Comparison of the accuracy and correctness of mortality estimates for Intensive Care Unit patients in internal clinics of the Czech Republic using APACHE II, APACHE IV, SAPS 3 and MPMoIII models
Aim To verify and compare the accuracies of mortality predictions in the Intensive Care Unit (ICU) of the Internal Clinic of Central Military Hospital in Prague, Czech Republic, using model APACHE II and the newer systems of the APACHE IV, SAPS 3 and MPMo III. Methods The data were collected retrospectively between 2011 and 2012, 1000 patients were evaluated. The assessment of the overall accuracy of the mortality predictions was performed using the standardized mortality ratio (SMR), and the calibration was assessed using the Lemeshow-Hosmer “goodness-of-fit” C statistic. Discrimination was evaluated using ROC curves based on calculations of the areas under the curve (AUCs). Results The APACHE II, SAPS 3, and MPMo III systems significantly overestimated the expected mortality, whereas the APACHE IV model led to correct estimations of the overall mortality. The discrimination capabilities of the models assessed according to the constructions of the ROC curves were evaluated as good, only the APACHE II was evaluated as satisfactory. The calibrations of all models were evaluated as unsatisfactory. Conclusion The best mortality estimation for the investigated population sample was provided by the APACHE IV system. The discrimination capabilities of all models for the studied population were satisfactory, but the calibration of all of the systems was unsatisfactory. The conclusions of our study are limited by the relatively small size of the investigated sample and the fact that this study was conducted at only a single site.
Keegan M, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med. 2011;163–9.
2.
Glance L, Osler T, Dick A. Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med. 2002;1976–82.
3.
Strand K, H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;467–78.
4.
Knaus W, Draper E, Wagner D, Zimmerman J. APACHE II: a severity of disease classification system. Crit Care Med. 1985;818–29.
5.
Higgins T, Kramer A, Nathanson B, Copes W, Stark M, Teres D. Prospective validation of the intensive care unit admission Mortality Probability Model (MPM0-III). Crit Care Med. 2009;1619–23.
6.
Metnitz P, Moreno R, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med. 2005;1336–44.
7.
Moreno R, Metnitz P, Almeida E, Jordan B, Bauer P, Campos R, et al. SAPS 3 Investigators. Intensive Care Med. 2005;1345–55.
8.
Zimmerman J, Kramer A, Mcnair D, Malila F. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;1297–310.
9.
Gall L, Loirat J, Alperovitch P, Glaser A, Granthil P, Mathieu C, et al. A simplified acute physiology score for ICU patients. Crit Care Med. 1984;975–7.
10.
Lemeshow S, Gall L, J. Modeling the severity of illness of ICU patients. A systems update. JAMA. 1994;1049–55.
11.
Teres D, Lemeshow S. Why severity models should be used with caution. Crit Care Clin. 1994;111–5.
12.
Higgins T, Teres D, Copes W, Nathanson B, Stark M, Kramer A. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med. 2007;827–35.
13.
Brier G. Verification of forecasts expressed in terms of probability. Mon Wea Rev. :1–3.
14.
Wu YC, Lee WC. Alternative Performance Measures for Prediction Models. PLoS One. 2014;91249.
15.
Lemeshow S, Hosmer D. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;92–106.
16.
Thygesen K, Alpert J, White H. Joint ESC/ ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction. Universal definition of myocardial infarction. J Am Coll Cardiol. 2007;2173–95.
17.
Widimský P, Budesínský T, Vorác D, Groch L, Zelízko M, Aschermann M, et al. PRAGUE’ Study Group Investigators. Long distance transport for primary angioplasty vs immediate thrombolysis in acute myocardial infarction. Final results of the randomized national multicentre trial--PRAGUE-2. Eur Heart J. 2003;94–104.
18.
Widimsky P, Zelizko M, Jansky P, Tousek F, Holm F, Aschermann M, et al. The incidence, treatment strategies and outcomes of acute coronary syndromes in the “reperfusion network” of different hospital types in the Czech Republic: results of the Czech evaluation of acute coronary syndromes in hospitalized patients (CZECH) registry. Int J Cardiol. 2007;212–9.
19.
Junior N, Mocelin A, Andrade A, Brauer F, Giannini L, Nunes F, et al. SAPS 3, APAC-HE IV or GRACE: which score to choose for acute coronary syndrome patients in intensive care units? São Paulo Med J Rev Paul Med. 2013;173–8.
20.
Costa E Silva V, De Castro I, Liaño F, Muriel A, Rodríguez-Palomares J, Yu L. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant. 2011;3894–901.
21.
Arabi Y, Haddad S, Goraj R, Al-Shimemeri A, Malik A, S. Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit. Crit Care. 2002;166–74.
22.
Kuzniewicz M, Vasilevskis E, Lane R, Dean M, Trivedi N, Rennie D, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest. 2008;1319–27.
23.
Keegan M, Gajic O, Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest. 2012;851–8.
24.
Katsaragakis S, Papadimitropoulos K, Antonakis P, Strergiopoulos S, Konstadoulakis M, Androulakis G. Comparison of Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score II (SAPS II) scoring systems in a single Greek intensive care unit. Crit Care Med. 2000;426–32.
25.
Cook D. Performance of APACHE III models in an Australian ICU. Chest. 2000;1732–8.
26.
Lee H, Shon YJ, Kim H, Paik H, Park HP. Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit. Korean J Anesthesiol. 2014;115–22.
27.
Murphy-Filkins R, Teres D, Lemeshow S, Hosmer D. Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med. 1996;1968–73.
28.
Beck D, Smith G, Taylor B. The impact of lowrisk intensive care unit admissions on mortality probabilities by SAPS II, APACHE II and APACHE III. Anaesthesia. 2002;21–6.
29.
Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, De Jonge E, Bosman R, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care. 2011;105-e111.
The statements, opinions and data contained in the journal are solely those of the individual authors and contributors and not of the publisher and the editor(s). We stay neutral with regard to jurisdictional claims in published maps and institutional affiliations.
