Free
Editorial Views  |   November 2010
Risk Stratification Index: An Important Advance in Comparing Health Care Apples to Oranges
Author Affiliations & Notes
  • Norman A. Cohen, M.D.
    **
  • Alexander A. Hannenberg, M.D.
    ††
  • **Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon. . ††Tufts University School of Medicine, Newton Wellesley Hospital, Newton, Massachusetts.
Article Information
Editorial Views / Practice Management / Quality Improvement
Editorial Views   |   November 2010
Risk Stratification Index: An Important Advance in Comparing Health Care Apples to Oranges
Anesthesiology 11 2010, Vol.113, 1004-1006. doi:10.1097/ALN.0b013e3181f79356
Anesthesiology 11 2010, Vol.113, 1004-1006. doi:10.1097/ALN.0b013e3181f79356
THE U.S. Congress, the Bush and Obama administrations, and the Centers for Medicare and Medicaid Services (CMS) have been committed to transforming Medicare into a “value-based” purchaser of services, requiring increasing amounts of transparency in medical data. This steady shift has produced a drive toward the public reporting of patient outcomes, as best exemplified by the government's hospital quality comparison Web site.1Hospitals and consumers both worry about the fairness and accuracy of the data presented—as well as the classic problem of the apples-to-oranges comparison.
The risk stratification method presented by Sessler et al  .1 is an enormous contribution to the quality and uniformity of hospital outcome reporting. Its superiority to existing tools, its demonstrated applicability beyond the Medicare population, and the ready access to methodology it provides all predict its widespread adoption as the national standard tool. The implications of accurately adjusting outcomes data are particularly relevant to patient access in complex and high-risk care. As the inherent financial consequences of public reporting grow, the likelihood of unintended incentives toward shunning the desperately ill, resulting from a failure to adequately reflect the inherent risks of such care in mortality data, becomes more serious. This benefit would directly accrue to our patients from application of such a method.
It is important to recognize that the public reporting of aggregate mortality data are much more an element of the financial incentive structure than a performance-improvement tool. Reports of all-cause mortality across all diagnoses and procedures, even when adequately adjusted for risk and severity, do little to inform hospitals and physicians about what is broken and where to invest resources dedicated to improvement. What such reports do is drive patient volume, and the accompanying dollars, toward better-performing centers. Notwithstanding the inescapable mathematical reality that the top 10% of centers cannot accommodate the remaining 90% of patients, these best practices should be recognized and disseminated.
Underperforming doctors and hospitals need to know their relative performance so they can improve. What do they need to know to respond to credible, risk-adjusted mortality figures that are substandard? It is possible that a pervasive weakness in systems and people exists and produces avoidable mortality. More likely, the providers will ask where the problems exist so that focused attention can be given to improving specific procedures and services. Adapting the Risk Stratification Index (RSI) by Sessler et al.  1 to allow reporting of similarly risk-adjusted observed and expected mortality at the procedure code level would provide more “actionable” data than aggregate, all-cause mortality. Development and validation of this capability should be a high priority. This adaptation moves the reporting process from the realm of the financial incentive toward a quality-improvement tool. The successful trials of the RSI formula in small volume samples are encouraging; this goal might be achievable.
Most notably, in our specialty, where anesthesia-related mortality has fallen so dramatically, our focus is on reduction of nonfatal consequences of anesthesia administration. Capturing a full range of adverse outcomes and understanding the factors predisposing patients to their occurrence is a compelling need—and perhaps more so in anesthesiology.
The capacity to drill down to specific services also addresses a shortcoming in physician performance measurement. The companion program to Hospital Compare is the Physician Quality Reporting Initiative.2Of the nearly 200 physician measures in this dataset, greater than 90% focus on process rather than quantifiable outcomes. The mammoth bias against outcome measures is based on the limitations to risk adjust and create truly comparable performance statistics. Can the RSI support the much-needed maturation of the Physician Quality Reporting Initiative system? To answer that question, and others related to RSI's role in current and future payment systems, one must understand the limitations imposed by the data on which RSI currently depends and evaluate the extensibility of RSI to other administrative datasets.
The database used for development and initial validation of the RSI is known as the Medicare Provider Analysis and Review (MEDPAR) file.3Every fee-for-service Medicare hospital admission has an entry in this system. Physician, outpatient hospital, and ambulatory surgery claims data appear in other administrative data files, each organized differently because each support markedly different payment rules. The differing file structures create challenges in linking facility, provider, and patient data together in a meaningful way. Furthermore, although the diagnosis code system is the same across these files, the procedural coding system used in MEDPAR—the International Classification of Diseases, Version 9-Clinical Modification, Volume 3—is far different from the Healthcare Common Procedure Coding System used in all the other datasets. The American Medical Association's Current Procedural Terminology® codes are an integral part of the Healthcare Common Procedure Coding System.
CMS pays for inpatient hospital care based on the principal diagnosis supporting admission, as adjusted for severity of illness and complexity. Because documented diagnoses drive this adjustment, hospitals have learned to do an excellent job abstracting clinical information into a relatively complete listing of diagnosis codes. For physicians, payment is driven by procedures performed—as reflected by diagnosis codes intended to justify and document medical necessity. Because patient complexity only rarely influences physician payment, physicians give much less attention to reporting a complete list of diagnosis codes with claims for payment. Consequently, applying the RSI method to current physician payment administrative claims data will be unlikely to generate a robust formula for predicting mortality, morbidity, or other endpoints—as Sessler et al.  1 demonstrate here with inpatient data. However, the reorganization of CMS contracting into combined part A and B Medicare Administrative Contractors has the potential to link patient-specific quality, cost, facility, and provider data in a way that could allow a modified RSI to include quality and cost inputs.
Sessler et al.  1 note that their method, including the Stepwise Hierarchical Selection process, can be modified to address morbidity measures as well as changes in the coding system. Given that hospitals will begin reporting diagnoses and procedures using the International Classification of Diseases, Version 10, by October 2013, that will be an essential requirement for ongoing applicability. The updated diagnosis coding system will have much greater specificity, but will be similar enough in structure that modifying RSI should be relatively straightforward. The International Classification of Diseases, Version 10, Procedure Coding System, is entirely new and will require a structurally different approach to collapsing procedure codes hierarchically. These changes will need to be validated, a task that will require time to collect sufficient administrative claims data to update RSI. After the transition, use of claims data for RSI purposes may need to wait until coders become fully proficient with International Classification of Diseases, Version 10, to avoid the “garbage in, garbage out” phenomenon.
We agree with Sessler et al.  1 that the RSI method is robust and flexible enough to overcome most if not all of these challenges. We also see opportunities to apply RSI now and in the immediate future.
Risk adjustment of payment for anesthesia care has long been part of the American Society of Anesthesiologists Relative Value Guide payment method through use of the Physical Status modifier although CMS does not recognize this modifier for payment. CMS may revisit risk-adjusted payment as it pursues “value-based purchasing” experiments in bundled payments4and accountable care organizations.5The capability of the RSI to quantify perioperative risk objectively may address the criticism leveled at the subjectivity of the physical status modifier.
In developing RSI, Sessler et al.  1 used MEDPAR data from 2001–2006. The authors note the difficulty in distinguishing diagnoses present on admission from those determined during hospital stay. Beginning in 2008, hospitals began reporting the occurrence of hospital-acquired conditions,6such as pressure ulcers or in-hospital falls and fractures, also indicating whether these conditions were present on admission. Having this data in more recent MEDPAR files will partially address the present on admission conundrum. Furthermore, RSI could be used to help assess risk for hospital-acquired conditions based on planned procedures and admitting diagnoses. Facilities could target interventions to reduce hospital-acquired conditions in patients predicted to be at high-risk through RSI. In addition, CMS could use RSI data to revise its punitive hospital-acquired conditions policy (i.e.  , no additional payment when conditions are present) to instead reward for reductions in risk-adjusted occurrence.
To put the achievement of Sessler et al.  1 in perspective, we must remember that they are working with a Medicare claims billing database, whose shortcomings are understandable because the dataset was never intended for quality assessment. Without applying validated risk adjustments to administrative data, erroneous and misleading conclusions will occur, as seen in a recent Health Affairs  study, and the implications for policy development driven by these data are obvious.2 The task of accurately reporting risks begs for the dissemination of electronic health records for perioperative care, the tool designed for this function and many others. Although the profession and the government are committed to adopting this technology, it is striking and disappointing that the role of the electronic health record in perioperative care is marginalized relative to the drive to deploy such systems in the office setting. The American Society of Anesthesiologists has invested heavily in the Anesthesia Quality Institute as a tool to aggregate and analyze outcomes, particularly nonfatal perioperative events, and ultimately give rise to risk-adjusted benchmarking. Perioperative electronic health records would enrich the Anesthesia Quality Institute's dataset, which would, in turn, allow far more sophisticated performance measurement and payment systems.
Improvements in quality and outcomes, and payment incentive reforms to promote these goals, require global and local investigations—looking both at the forest and the trees. RSI will help better answer the global questions, particularly if the refinements and expansions described here prove to be both possible and valid. The local questions will be answered only through iterative changes in processes measured and tracked prospectively, a job best done by electronic health records and data registries such as the Anesthesia Quality Institute. Sessler et al.  1 deserve commendation for bringing the forest into sharper focus, moving us one large step closer to comparing the apples and oranges of healthcare outcomes.
References
Sessler DI, Sigl JC, Manberg PJ, Kelley SD, Schubert A, Chamoun NG: Broadly applicable risk stratification system for predicting duration of hospitalization and mortality. Anesthesiology 2010; 113:1026–37Sessler, DI Sigl, JC Manberg, PJ Kelley, SD Schubert, A Chamoun, NG
Dulisse B, Cromwell J: No harm found when nurse anesthetists work without supervision by physicians. Health Aff 2010; 29:1469–75Dulisse, B Cromwell, J