Correspondence  |   August 2016
In Reply
Author Notes
  • Division of General Surgery, University of Toronto, Toronto, Ontario, Canada (E.W.).
  • James C. Eisenach, M.D., served as Editor-in-Chief for this exchange.
    James C. Eisenach, M.D., served as Editor-in-Chief for this exchange.×
  • (Accepted for publication March 30, 2016.)
    (Accepted for publication March 30, 2016.)×
Article Information
Correspondence   |   August 2016
In Reply
Anesthesiology 8 2016, Vol.125, 423. doi:10.1097/ALN.0000000000001179
Anesthesiology 8 2016, Vol.125, 423. doi:10.1097/ALN.0000000000001179
We appreciate the thoughtful attention that Dr. Hofer and colleagues have given our article describing the advantages and disadvantages of reliability, or “shrinkage” adjustment.1  Their title, “Current Quality Registries Lack the Accurate Data Needed to Perform Adequate Reliability Adjustments” may be accurate for the anesthesia data collections they mention but not for all surgical quality registries.
Dr. Hofer and colleagues’ message about the importance of measurement error cannot be understated. Measurement error, whether in administrative data or in registries, undermines both the validity and utility of quality measurement. When reliability adjustment is applied, unmeasured patient and case-mix factors leave “residual” variation that may be falsely attributed to hospitals or physicians rather than inadequate risk adjustment.2  With or without reliability adjustment, measurement error is critical when benchmarking quality across hospitals or physicians because federal mandates are linking payment to outcome-based performance measurement.
Physicians and hospital leaders already appreciate that meaningful outcomes comparisons are very costly to produce, particularly when accrued through a clinical registry and analyzed with the necessary statistical expertise. The first question is whether physicians believe that meaningful outcomes comparisons are important enough to pay for them. The American College of Surgeons (ACS) and the Society of Thoracic Surgeons (STS) staked out their positions on this issue decades ago and currently generate the highest-quality outcomes data in surgery while stewarding multiple measures in the National Quality Forum.3–5  Some may wonder why anesthesiologists have not taken a similar leadership position.6  However, it is important to consider that (1) participation in these registries is costly, (2) neither the ACS nor STS registry outcomes are part of current or proposed Centers for Medicare and Medicaid Services payment programs, and (3) the jury is still out on whether participation in ACS or STS registries improves quality.7,8 
So how can anesthesiologists improve the quality of quality measurement? This is crucial because mandated links between payment and “performance” are moving forward with or (more commonly) without high-quality measurement science. Solutions are many: investing in anesthesia registries, fostering partnerships with surgeons to share the costs of registries, and uniting with surgeons and nurses for a stronger political voice. In brief, anesthesiologists must either “pony up” the financial and leadership costs of performance measurement or risk being left in the dust.
Competing Interests
The authors declare no competing interests.
Elliot Wakeam, M.D., M.P.H., Joseph A. Hyder, M.D., Ph.D. Division of General Surgery, University of Toronto, Toronto, Ontario, Canada (E.W.).
Wakeam, E, Hyder, JA Reliability of reliability adjustment for quality improvement and value-based payment.. Anesthesiology. (2016). 124 16–8 [Article] [PubMed]
Lawson, EH, Ko, CY, Adams, JL, Chow, WB, Hall, BL Reliability of evaluating hospital quality by colorectal surgical site infection type.. Ann Surg. (2013). 258 994–1000 [Article] [PubMed]
Shahian, DM, Jacobs, JP, Edwards, FH, Brennan, JM, Dokholyan, RS, Prager, RL, Wright, CD, Peterson, ED, McDonald, DE, Grover, FL The society of thoracic surgeons national database.. Heart. (2013). 99 1494–501 [Article] [PubMed]
Cohen, ME, Ko, CY, Bilimoria, KY et al.  Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.. J Am Coll Surg. (2013). 217 336–46 e331 [Article] [PubMed]
Hyder, JA, Roy, N, Wakeam, E, Hernandez, R, Kim, SP, Bader, AM, Cima, RR, Nguyen, LL Performance measurement in surgery through the National Quality Forum.. J Am Coll Surg. (2014). 219 1037–46 [Article] [PubMed]
Hyder, JA, Niconchuk, J, Glance, LG, Neuman, MD, Cima, RR, Dutton, RP, Nguyen, LL, Fleisher, LA, Bader, AM What can the national quality forum tell us about performance measurement in anesthesiology?. Anesth Analg. (2015). 120 440–8 [Article] [PubMed]
Osborne, NH, Nicholas, LH, Ryan, AM, Thumma, JR, Dimick, JB Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries.. JAMA. (2015). 313 496–504 [Article] [PubMed]
Etzioni, DA, Wasif, N, Dueck, AC, Cima, RR, Hohmann, SF, Naessens, JM, Mathur, AK, Habermann, EB Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality.. JAMA. (2015). 313 505–11 [Article] [PubMed]