Correspondence  |   May 2016
Predilection for Poor Prediction with the Surgical Apgar Score
Author Notes
  • Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota.
  • (Accepted for publication January 28, 2016.)
    (Accepted for publication January 28, 2016.)×
Article Information
Correspondence   |   May 2016
Predilection for Poor Prediction with the Surgical Apgar Score
Anesthesiology 5 2016, Vol.124, 1195-1196. doi:10.1097/ALN.0000000000001066
Anesthesiology 5 2016, Vol.124, 1195-1196. doi:10.1097/ALN.0000000000001066
To the Editor:
I enjoyed the recent article by Terekhov et al.,1  “Preoperative Surgical Risk Predictions Are Not Meaningfully Improved by Including the Surgical Apgar Score” (SAS). I value the contributions of these authors to this field of investigation, including their pioneering work with the SAS.2  The authors made two methodological choices that may have contributed to the study concluding no improvement in prediction, so I humbly offer two suggestions to permit a more definitive test of their hypothesis.
First, would the authors consider performing their analyses using an alternative sampling interval for vital signs? The authors constructed the vital signs components of the SAS, lowest heart rate, and lowest mean arterial pressure, using “instantaneous” measures, or the true lowest heart rate and lowest mean arterial pressure in the record. These “instantaneous” values for the SAS are the least useful option for predicting outcomes when compared with alternatives such as moving median values over 5- and 10-min windows.3  In essence, the choice of instantaneous values biases the assessment to no benefit of the SAS.
Second, would the authors consider adding a calculation of risk reclassification to better test the clinical utility of the SAS? The authors reported the c-statistic and Brier score to evaluate the utility of the SAS. Although statistically robust, neither of these measures provides clinical insight. Moreover, the c-statistic is known to change minimally even when important improvements are made with risk prediction.4  For this reason, the use of a reclassification measure may be applied to provide a more clinically meaningful assessment of change in risk prediction.5  Reclassification approaches can be problematic, but the concept of categorizing patients into high- and low-risk groups is clinically intuitive and actionable, because we treat high-risk patients differently such as with admission to the intensive care unit.
The potential for real-time risk revision is not known, and with these suggestions, the authors may be able to more robustly test its potential.
Competing Interests
Although the author is a current awardee of the Anesthesia Patient Safety Foundation and Anesthesia Quality Institute, this letter was written on separate time. “No Funding Received” is the most accurate description of the funding details. The author receives no funding from industry or honoraria and has no conflicts of interest to disclose.
Joseph A. Hyder, M.D., Ph.D., Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota.
Terekhov, MA, Ehrenfeld, JM, Wanderer, JP Preoperative surgical risk predictions are not meaningfully improved by including the Surgical Apgar Score: An analysis of the risk quantification index and present-on-admission risk models.. Anesthesiology. (2015). 123 1059–66 [Article] [PubMed]
Regenbogen, SE, Ehrenfeld, JM, Lipsitz, SR, Greenberg, CC, Hutter, MM, Gawande, AA Utility of the surgical Apgar score: Validation in 4119 patients.. Arch Surg. (2009). 144 30–6; discussion 37 [Article] [PubMed]
Hyder, JA, Kor, DJ, Cima, RR, Subramanian, A How to improve the performance of intraoperative risk models: An example with vital signs using the surgical Apgar score.. Anesth Analg. (2013). 117 1338–46 [Article] [PubMed]
Cook, NR Use and misuse of the receiver operating characteristic curve in risk prediction.. Circulation. (2007). 115 928–35 [Article] [PubMed]
Leening, MJ, Vedder, MM, Witteman, JC, Pencina, MJ, Steyerberg, EW Net reclassification improvement: Computation, interpretation, and controversies: A literature review and clinician’s guide.. Ann Intern Med. (2014). 160 122–31 [Article] [PubMed]