Comparison of the accuracy and correctness of mortality estimates for Intensive Care Unit patients in internal clinics of the Czech Republic using APACHE II, APACHE IV, SAPS 3 and MPMoIII models
Aim To verify and compare the accuracies of mortality predictions in the Intensive Care Unit (ICU) of the Internal Clinic of Central Military Hospital in Prague, Czech Republic, using model APACHE II and the newer systems of the APACHE IV, SAPS 3 and MPMo III. Methods The data were collected retrospectively between 2011 and 2012, 1000 patients were evaluated. The assessment of the overall accuracy of the mortality predictions was performed using the standardized mortality ratio (SMR), and the calibration was assessed using the Lemeshow-Hosmer “goodness-of-fit” C statistic. Discrimination was evaluated using ROC curves based on calculations of the areas under the curve (AUCs). Results The APACHE II, SAPS 3, and MPMo III systems significantly overestimated the expected mortality, whereas the APACHE IV model led to correct estimations of the overall mortality. The discrimination capabilities of the models assessed according to the constructions of the ROC curves were evaluated as good, only the APACHE II was evaluated as satisfactory. The calibrations of all models were evaluated as unsatisfactory. Conclusion The best mortality estimation for the investigated population sample was provided by the APACHE IV system. The discrimination capabilities of all models for the studied population were satisfactory, but the calibration of all of the systems was unsatisfactory. The conclusions of our study are limited by the relatively small size of the investigated sample and the fact that this study was conducted at only a single site.
Lemeshow S, Hosmer D. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;92–106.
2.
Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, De Jonge E, Bosman R, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care. 2011;105-e111.
3.
Beck D, Smith G, Taylor B. The impact of lowrisk intensive care unit admissions on mortality probabilities by SAPS II, APACHE II and APACHE III. Anaesthesia. 2002;21–6.
4.
Murphy-Filkins R, Teres D, Lemeshow S, Hosmer D. Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med. 1996;1968–73.
5.
Lee H, Shon YJ, Kim H, Paik H, Park HP. Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit. Korean J Anesthesiol. 2014;115–22.
6.
Cook D. Performance of APACHE III models in an Australian ICU. Chest. 2000;1732–8.
7.
Katsaragakis S, Papadimitropoulos K, Antonakis P, Strergiopoulos S, Konstadoulakis M, Androulakis G. Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit. Crit Care Med. 2000;426–32.
8.
Keegan M, Gajic O, Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest. 2012;851–8.
9.
Kuzniewicz M, Vasilevskis E, Lane R, Dean M, Trivedi N, Rennie D, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest. 2008;1319–27.
10.
Arabi Y, Haddad S, Goraj R, Al-Shimemeri A, Malik A, S. Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit. Crit Care. 2002;166–74.
11.
Costa E Silva V, De Castro I, Liaño F, Muriel A, Rodríguez-Palomares J, Yu L. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant. 2011;3894–901.
12.
Junior N, Mocelin A, Andrade A, Brauer F, Giannini L, Nunes F, et al. SAPS 3, APAC-HE IV or GRACE: which score to choose for acute coronary syndrome patients in intensive care units? São Paulo Med J Rev Paul Med. 2013;173–8.
13.
Widimsky P, Zelizko M, Jansky P, Tousek F, Holm F, Aschermann M, et al. The incidence, treatment strategies and outcomes of acute coronary syndromes in the “reperfusion network” of different hospital types in the Czech Republic: results of the Czech evaluation of acute coronary syndromes in hospitalized patients (CZECH) registry. Int J Cardiol. 2007;212–9.
14.
Widimský P, Budesínský T, Vorác D, Groch L, Zelízko M, Aschermann M, et al. PRAGUE’ Study Group Investigators. Long distance transport for primary angioplasty vs immediate thrombolysis in acute myocardial infarction. Final results of the randomized national multicentre trial--PRAGUE-2. Eur Heart J. 2003;94–104.
15.
Thygesen K, Alpert J, White H. Joint ESC/ ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction. Universal definition of myocardial infarction. J Am Coll Cardiol. 2007;2173–95.
16.
Keegan M, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med. 2011;163–9.
17.
Wu YC, Lee WC. Alternative Performance Measures for Prediction Models. PLoS One. 2014;91249.
18.
Brier G. Verification of forecasts expressed in terms of probability. Mon Wea Rev. :1–3.
19.
Higgins T, Teres D, Copes W, Nathanson B, Stark M, Kramer A. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med. 2007;827–35.
20.
Teres D, Lemeshow S. Why severity models should be used with caution. Crit Care Clin. 1994;111–5.
21.
Lemeshow S, Gall L, J. Modeling the severity of illness of ICU patients. A systems update. JAMA. 1994;1049–55.
22.
Gall L, Loirat J, Alperovitch P, Glaser A, Granthil P, Mathieu C, et al. A simplified acute physiology score for ICU patients. Crit Care Med. 1984;975–7.
23.
Zimmerman J, Kramer A, Mcnair D, Malila F. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;1297–310.
24.
Moreno R, Metnitz P, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med. 2005;1345–55.
25.
Metnitz P, Moreno R, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med. 2005;1336–44.
26.
Higgins T, Kramer A, Nathanson B, Copes W, Stark M, Teres D. Prospective validation of the intensive care unit admission Mortality Probability Model (MPM0-III). Crit Care Med. 2009;1619–23.
27.
Knaus W, Draper E, Wagner D, Zimmerman J. APACHE II: a severity of disease classification system. Crit Care Med. 1985;818–29.
28.
Strand K, H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;467–78.
29.
Glance L, Osler T, Dick A. Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med. 2002;1976–82.
The statements, opinions and data contained in the journal are solely those of the individual authors and contributors and not of the publisher and the editor(s). We stay neutral with regard to jurisdictional claims in published maps and institutional affiliations.
