Comparison of the accuracy and correctness of mortality estimates for Intensive Care Unit patients in internal clinics of the Czech Republic using APACHE II, APACHE IV, SAPS 3 and MPMoIII models
Aim To verify and compare the accuracies of mortality predictions in the Intensive Care Unit (ICU) of the Internal Clinic of Central Military Hospital in Prague, Czech Republic, using model APACHE II and the newer systems of the APACHE IV, SAPS 3 and MPMo III. Methods The data were collected retrospectively between 2011 and 2012, 1000 patients were evaluated. The assessment of the overall accuracy of the mortality predictions was performed using the standardized mortality ratio (SMR), and the calibration was assessed using the Lemeshow-Hosmer “goodness-of-fit” C statistic. Discrimination was evaluated using ROC curves based on calculations of the areas under the curve (AUCs). Results The APACHE II, SAPS 3, and MPMo III systems significantly overestimated the expected mortality, whereas the APACHE IV model led to correct estimations of the overall mortality. The discrimination capabilities of the models assessed according to the constructions of the ROC curves were evaluated as good, only the APACHE II was evaluated as satisfactory. The calibrations of all models were evaluated as unsatisfactory. Conclusion The best mortality estimation for the investigated population sample was provided by the APACHE IV system. The discrimination capabilities of all models for the studied population were satisfactory, but the calibration of all of the systems was unsatisfactory. The conclusions of our study are limited by the relatively small size of the investigated sample and the fact that this study was conducted at only a single site.
Lemeshow S, Hosmer D. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982:92–106.
2
Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, De Jonge E, Bosman R, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care 2011:105-e111.
3
Beck D, Smith G, Taylor B. The impact of lowrisk intensive care unit admissions on mortality probabilities by SAPS II, APACHE II and APACHE III. Anaesthesia 2002:21–6.
4
Murphy-Filkins R, Teres D, Lemeshow S, Hosmer D. Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med 1996:1968–73.
5
Lee H, Shon Y-J, Kim H, Paik H, Park H-P. Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit. Korean J Anesthesiol 2014:115–22.
6
Cook D. Performance of APACHE III models in an Australian ICU. Chest 2000:1732–8.
7
Katsaragakis S, Papadimitropoulos K, Antonakis P, Strergiopoulos S, Konstadoulakis M, Androulakis G. Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit. Crit Care Med 2000:426–32.
8
Keegan M, Gajic O, Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest 2012:851–8.
9
Kuzniewicz M, Vasilevskis E, Lane R, Dean M, Trivedi N, Rennie D, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest 2008:1319–27.
10
Arabi Y, Haddad S, Goraj R, Al-Shimemeri A, Malik A, S. Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit. Crit Care 2002:166–74.
11
Costa E Silva V, De Castro I, Liaño F, Muriel A, Rodríguez-Palomares J, Yu L. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant 2011:3894–901.
12
Junior N, Mocelin A, Andrade A, Brauer F, Giannini L, Nunes F, et al. SAPS 3, APAC-HE IV or GRACE: which score to choose for acute coronary syndrome patients in intensive care units? São Paulo Med J Rev Paul Med 2013:173–8.
13
Widimsky P, Zelizko M, Jansky P, Tousek F, Holm F, Aschermann M, et al. The incidence, treatment strategies and outcomes of acute coronary syndromes in the “reperfusion network” of different hospital types in the Czech Republic: results of the Czech evaluation of acute coronary syndromes in hospitalized patients (CZECH) registry. Int J Cardiol 2007:212–9.
14
Widimský P, Budesínský T, Vorác D, Groch L, Zelízko M, Aschermann M, et al. PRAGUE’ Study Group Investigators. Long distance transport for primary angioplasty vs immediate thrombolysis in acute myocardial infarction. Final results of the randomized national multicentre trial--PRAGUE-2. Eur Heart J 2003:94–104.
15
Thygesen K, Alpert J, White H. Joint ESC/ ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction. Universal definition of myocardial infarction. J Am Coll Cardiol 2007:2173–95.
16
Keegan M, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med 2011:163–9.
17
Wu Y-C, Lee W-C. Alternative Performance Measures for Prediction Models. PLoS One 2014:91249.
18
Brier G. Verification of forecasts expressed in terms of probability. Mon Wea Rev n.d.:1–3.
19
Higgins T, Teres D, Copes W, Nathanson B, Stark M, Kramer A. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007:827–35.
20
Teres D, Lemeshow S. Why severity models should be used with caution. Crit Care Clin 1994:111–5.
21
Lemeshow S, Gall L, J. Modeling the severity of illness of ICU patients. A systems update. JAMA 1994:1049–55.
22
Gall L, Loirat J, Alperovitch P, Glaser A, Granthil P, Mathieu C, et al. A simplified acute physiology score for ICU patients. Crit Care Med 1984:975–7.
23
Zimmerman J, Kramer A, Mcnair D, Malila F. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006:1297–310.
24
Moreno R, Metnitz P, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med 2005:1345–55.
25
Metnitz P, Moreno R, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med 2005:1336–44.
26
Higgins T, Kramer A, Nathanson B, Copes W, Stark M, Teres D. Prospective validation of the intensive care unit admission Mortality Probability Model (MPM0-III). Crit Care Med 2009:1619–23.
27
Knaus W, Draper E, Wagner D, Zimmerman J. APACHE II: a severity of disease classification system. Crit Care Med 1985:818–29.
28
Strand K, H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand 2008:467–78.
29
Glance L, Osler T, Dick A. Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med 2002:1976–82.
The statements, opinions and data contained in the journal are solely those of the individual authors and contributors and not of the publisher and the editor(s). We stay neutral with regard to jurisdictional claims in published maps and institutional affiliations.
