The effect of disease-prevalence adjustments on the accuracy of a logistic prediction model.
The accuracy of a logistic prediction model is degraded when it is transported to populations with outcome prevalences different from that of the population used to derive the model. The resultant errors can have major clinical implications. Accordingly, the authors developed a logistic prediction model with respect to the noninvasive diagnosis of coronary disease based on 1,824 patients who underwent exercise testing and coronary angiography, varied the prevalence of disease in various "test" populations by random sampling of the original "derivation" population, and determined the accuracy of the logistic prediction model before and after the application of a mathematical algorithm designed to adjust only for these differences in prevalence. The accuracy of each prediction model was quantified in terms of receiver operating characteristic (ROC) curve area (discrimination) and chi-square goodness-of-fit (calibration). As the prevalence of the test population diverged from the prevalence of the derivation population, discrimination improved (ROC-curve areas increased from 0.82 +/- 0.02 to 0.87 +/- 0.03; p