- New measures: Variants of the c statistics for survival, reclassification tables, NR, IDI
- Decision analytics measures : “decision curves” by making decisions based on model predictions
- Validation in fully independent, external data is the best way to compare the performance of a model with and without a new marker
- Nagelkerke’s R2 test for logarithm predictions is based on the difference in the -2 log likelihood of a model and a model with one or more predictors
- Scaled Brier Score very similar to the Pearson’s R2 statistic
- C statistic is related to Somer’s D
- A popular extension of the C statistics with censored data can be obtained by ignoring the pairs that cannot be ordered
- Addition to the c statistics is the discriminatory slope
- Smoothing technique: Loess algorithm
- Recalibration framework proposed by Cox
Cook proposed a reclassification test as a variant of the Hosmer-Lemeshow statistic within the reclassified categories leading to the CH2 statistic (idea extended by Pencia et al by conditioning on the outcome) - Net classification Improvement (NRI)
- Youden index implies weighting by the non-events odds
- Documentation of decision-curve analysis can be found at: http://www.decisioncurveanalysis.org
- Cook’s reclassification test
- Reclassification can be assessed using a scatter plot before and after change in the model.
- Use design library to generate good plots in R
- Recalibration parameters as proposed by Cox (intercept and calibration slope) are more informative
- Key information for comparing performances of 2 models is contained in the margins of the reclassification tables
