Newly Published
Perioperative Medicine  |   February 2017
Validation and Calibration of the Risk Stratification Index
Author Notes
  • From the Lown Institute, Boston, Massachusetts (G.F.C., N.G.C., V.S.); Department of Environmental Health, Harvard T.H. Chan School of Public Health, Boston, Massachusetts (L.L.); and Department of Outcomes Research, Anesthesiology Institute, Cleveland Clinic, Cleveland, Ohio (D.I.S.).
  • Submitted for publication August 19, 2016. Accepted for publication January 4, 2017.
    Submitted for publication August 19, 2016. Accepted for publication January 4, 2017.×
  • Research Support: Supported by the Lown Institute, Boston, Massachusetts.
    Research Support: Supported by the Lown Institute, Boston, Massachusetts.×
  • Competing Interests: The authors declare no competing interests.
    Competing Interests: The authors declare no competing interests.×
  • Correspondence: Address correspondence to Dr. Sessler: Department of Outcomes Research, Anesthesiology Institute, Cleveland Clinic, 9500 Euclid Ave—P77, Cleveland, Ohio 44195. DS@OR.org. Information on purchasing reprints may be found at www.anesthesiology.org or on the masthead page at the beginning of this issue. Anesthesiology’s articles are made freely accessible to all readers, for personal use only, 6 months from the cover date of the issue.
Article Information
Perioperative Medicine / Practice Management
Perioperative Medicine   |   February 2017
Validation and Calibration of the Risk Stratification Index
Anesthesiology Newly Published on February 16, 2017. doi:10.1097/ALN.0000000000001560
Anesthesiology Newly Published on February 16, 2017. doi:10.1097/ALN.0000000000001560
Abstract

Background: The Risk Stratification Index was developed from 35 million Medicare hospitalizations from 2001 to 2006 but has yet to be externally validated on an independent large national data set, nor has it been calibrated. Finally, the Medicare Analysis and Provider Review file now allows 25 rather than 9 diagnostic codes and 25 rather than 6 procedure codes and includes present-on-admission flags. The authors sought to validate the index on new data, test the impact of present-on-admission codes, test the impact of the expansion to 25 diagnostic and procedure codes, and calibrate the model.

Methods: The authors applied the original index coefficients to 39,753,036 records from the 2007–2012 Medicare Analysis data set and calibrated the model. The authors compared their results with 25 diagnostic and 25 procedure codes, with results after restricting the model to the first 9 diagnostic and 6 procedure codes and to codes present on admission.

Results: The original coefficients applied to the 2007–2012 data set yielded C statistics of 0.83 for 1-yr mortality, 0.84 for 30-day mortality, 0.94 for in-hospital mortality, and 0.86 for median length of stay—values nearly identical to those originally reported. Calibration equations performed well against observed outcomes. The 2007–2012 model discriminated similarly when codes were restricted to nine diagnostic and six procedure codes. Present-on-admission models were about 10% less predictive for in-hospital mortality and hospital length of stay but were comparably predictive for 30-day and 1-yr mortality.

Conclusions: Risk stratification performance was largely unchanged by additional diagnostic and procedure codes and only slightly worsened by restricting analysis to codes present on admission. The Risk Stratification Index, after calibration, thus provides excellent discrimination and calibration for important health services outcomes and thus appears to be a good basis for making hospital comparisons.