Seminars in Oncology
Volume 37, Issue 1 , Pages 31-38 , February 2010

Traditional Statistical Methods for Evaluating Prediction Models Are Uninformative as to Clinical Value: Towards a Decision Analytic Framework

  • Andrew J. Vickers

      Affiliations

    • Corresponding Author InformationAddress correspondence to Andrew J. Vickers, PhD, Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, 1275 York Ave, Box 44, New York, NY 10065
    • ,
  • Angel M. Cronin

References 

  1. Murphy NC, Biankin AV, Millar EK, et al. Loss of STARD10 expression identifies a group of poor prognosis breast cancers independent of HER2/Neu and triple negative status. Int J Cancer. August 12, 2009;Epub ahead of print
  2. Korse CM, Taal BG, de Groot CA, Bakker RH, Bonfrer JM. Chromogranin-A and N-terminal pro-brain natriuretic peptide: an excellent pair of biomarkers for diagnostics in patients with neuroendocrine tumor. J Clin Oncol. 2009;27:4293–4299
  3. Garcia-Albeniz X, Gallego R. Prognostic role of plasma insulin-like growth factor (IGF) and IGF-binding protein 3 in metastatic colorectal cancer. Clin Cancer Res. 2009;15:5288
  4. Stinchcombe TE, Hodgson L, Herndon JE, et al. Treatment outcomes of different prognostic groups of patients on Cancer and Leukemia Group B trial 39801: induction chemotherapy followed by chemoradiotherapy compared with chemoradiotherapy alone for unresectable stage III non-small cell lung cancer. J Thorac Oncol. 2009;4:1117–1125
  5. Warren M, Venner PM, North S, et al. A population-based study examining the effect of tyrosine kinase inhibitors on survival in metastatic renal cell carcinoma in Alberta and the role of nephrectomy prior to treatment. Can Urol Assoc J. 2009;3:281–289
  6. Cooperberg MR, Hinotsu S, Namiki M, et al. Risk assessment among prostate cancer patients receiving primary androgen deprivation therapy. J Clin Oncol. 2009;27:4306–4313
  7. Monzon JG, Cremin C, Armstrong L, et al. Validation of predictive models for germline mutations in DNA mismatch repair genes in colorectal cancer. Int J Cancer. 2009;27:4555–4562
  8. Federico M, Bellei M, Marcheselli L, et al. Follicular Lymphoma International Prognostic Index 2: a new prognostic index for follicular lymphoma developed by the International Follicular Lymphoma Prognostic Factor Project. J Clin Oncol. 2009;27:4555–4562
  9. Shariat SF, Karakiewicz PI, Roehrborn CG, et al. An updated catalog of prostate cancer predictive tools. Cancer. 2008;113:3075–3099
  10. Thompson IM, Ankerst DP, Chi C, et al. Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst. 2006;98:529–534
  11. Olivotto IA, Bajdik CD, Ravdin PM, et al. Population-based validation of the prognostic model ADJUVANT! for early breast cancer. J Clin Oncol. 2005;23:2716–2725
  12. Marchionni L, Wilson RF, Wolff AC, et al. Systematic review: gene expression profiling assays in early-stage breast cancer. Ann Intern Med. 2008;148:358–369
  13. Sheridan S, Pignone M, Mulrow C. Framingham-based tools to calculate the global risk of coronary heart disease: a systematic review of tools for clinicians. J Gen Intern Med. 2003;18:1039–1052
  14. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818–829
  15. Parody R, Martino R, Sanchez F, Subira M, Hidalgo A, Sierra J. Predicting survival in adults with invasive aspergillosis during therapy for hematological malignancies or after hematopoietic stem cell transplantation: single-center analysis and validation of the Seattle, French, and Strasbourg prognostic indexes. Am J Hematol. 2009;84:571–578
  16. Nunes RA, Vale RG, Simao R, et al. Prediction of Vo2max during cycle ergometry based on submaximal ventilatory indicators. J Strength Cond Res. 2009;23:1745–1751
  17. Schmid-Mohler G, Thut MP, Wuthrich RP, Denhaerynck K, De Geest S. Non-adherence to immunosuppressive medication in renal transplant recipients within the scope of the integrative model of behavioral prediction: a cross-sectional study. Clin Transplant. Aug 11, 2009;Epub ahead of print
  18. Goldraich L, Beck-da-Silva L, Clausell N. Are scores useful in advanced heart failure?. Expert Rev Cardiovasc Ther. 2009;7:985–997
  19. Koenig W, Vossen CY, Mallat Z, Brenner H, Benessiano J, Rothenbacher D. Association between type II secretory phospholipase A2 plasma concentrations and activity and cardiovascular events in patients with coronary heart disease. Eur Heart J. 2009;30:2742–2748
  20. Verhoeven CJ, Oudenaarden A, Hermus MA, Porath MM, Oei SG, Mol BW. Validation of models that predict cesarean section after induction of labor. Ultrasound Obstet Gynecol. 2009;34:316–321
  21. Law LW, Leung TY, Sahota DS, Chan LW, Fung TY, Lau TK. Which ultrasound or biochemical markers are independent predictors of small-for-gestational age?. Ultrasound Obstet Gynecol. 2009;34:283–287
  22. Massicotte L, Capitanio U, Beaulieu D, Roy JD, Roy A, Karakiewicz PI. Independent validation of a model predicting the need for packed red blood cell transfusion at liver transplantation. Transplantation. 2009;88:386–391
  23. Kosmider O, Gelsi-Boyer V, Cheok M, et al. TET2 mutation is an independent favorable prognostic factor in myelodysplastic syndromes (MDS). Blood. 2009;114:3285–3291
  24. Lin CL, Lin PH, Chou LW, et al. Model-based prediction of length of stay for rehabilitating stroke patients. J Formos Med Assoc. 2009;108:653–662
  25. Kattan MW. Judging new markers by their ability to improve predictive accuracy. J Natl Cancer Inst. 2003;95:634–635
  26. Kattan MW, Eastham JA, Stapleton AM, Wheeler TM, Scardino PT. A preoperative nomogram for disease recurrence following radical prostatectomy for prostate cancer. J Natl Cancer Inst. 1998;90:766–771
  27. Kuijpers T, van der Heijden GJ, Vergouwe Y, et al. Good generalizability of a prediction rule for prediction of persistent shoulder pain in the short term. J Clin Epidemiol. 2007;60:947–953
  28. Grover SA, Hemmelgarn B, Joseph L, Milot A, Tremblay G. The role of global risk assessment in hypertension therapy. Can J Cardiol. 2006;22:606–613
  29. Zorn KC, Capitanio U, Jeldres C, et al. Multi-institutional external validation of seminal vesicle invasion nomograms: head-to-head comparison of Gallina nomogram versus 2007 Partin tables. Int J Radiat Oncol Biol Phys. 2009;73:1461–1467
  30. Tice JA, Cummings SR, Smith-Bindman R, Ichikawa L, Barlow WE, Kerlikowske K. Using clinical factors and mammographic breast density to estimate breast cancer risk: development and validation of a new predictive model. Ann Intern Med. 2008;148:337–347
  31. Janes H, Pepe MS, Gu W. Assessing the value of risk predictions by using risk stratification tables. Ann Intern Med. 2008;149:751–760
  32. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565–574
  33. Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53

 Supported in part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and P50-CA92629 SPORE grant from the National Cancer Institute to Dr P.T. Scardino.

 The authors have no primary financial relationships with any companies directly interested in the subject matter of this manuscript.

PII: S0093-7754(09)00229-2

doi: 10.1053/j.seminoncol.2009.12.004

Seminars in Oncology
Volume 37, Issue 1 , Pages 31-38 , February 2010

Article Tools

* Image Usage & Resolution