Editorial Views  |   May 2002
Perioperative Risk: How Can We Study the Influence of Provider Characteristics?
Author Affiliations & Notes
  • Lee A. Fleisher, M.D.
  • Gerard F. Anderson, Ph.D.
  • *Associate Professor of Anesthesiology and Vice Chairman of Clinical Investigation, Joint Appointments in Medicine and Health Policy and Management, †Professor and Associate Chair of Health Policy & Management, Joint Appointment in Medicine and International Health, Johns Hopkins Medical Institutions, Baltimore, Maryland.
Article Information
Editorial Views
Editorial Views   |   May 2002
Perioperative Risk: How Can We Study the Influence of Provider Characteristics?
Anesthesiology 5 2002, Vol.96, 1039-1041. doi:
Anesthesiology 5 2002, Vol.96, 1039-1041. doi:
IN the 20th century, the medical community was intensely interested in understanding the factors associated with perioperative risk. In the current issue of Anesthesiology, Silber et al.  1 have added to this literature and studied the influence of anesthesia provider characteristics on outcome.
One of the earliest systematic analyses of the factors associated with perioperative risk occurred in 1935 when Ruth helped to establish a commission to analyze perioperative deaths. 2 The commission relied on voluntary submission of cases and determined the cause of death by majority vote. This methodology was deemed inadequate in subsequent studies.
A major advance in the analysis of perioperative risk was the report by Beecher and Todd 3 of the factors associated with anesthetic death in 10 institutions, published in 1954. Their study reviewed the administration of 599,548 anesthetics to patients with a wide range of diseases. The cause of mortality was determined at the local institution by consensus of a surgeon and the chief anesthetist of the institution. Although the methodology for assigning the cause of death was criticized as being arbitrary, the major contributions were the recognition that there are three major causes of death in the perioperative period and the quantification of these risks. They found that a patient had an overall chance of mortality of 1 in 75 cases in the perioperative period. Anesthesia was reported as the primary cause of mortality in 1 in 2,680 cases, and was either the primary or contributory cause of mortality in 1 in 1,560 cases. Surgical error in diagnosis, judgment, or technique was the primary cause of death in 1 in 420 cases, and patient disease was the primary cause in 1 in 95 cases.
Subsequent researchers have adopted this framework of the tripartite components of perioperative risk (surgery, anesthesia, patient disease). 4–6 Because it is the major cause of mortality, most of the subsequent analyses have focused appropriately on patient disease. Specifically, investigators have focused on identifying medical conditions associated with poor outcomes and have attempted to identify specific interventions that will improve outcome. 7–10 The relatively high probability of death associated with patient disease has allowed the majority of these studies to be performed in a single center or a small group of centers.
Although most of the research has focused on identifying and minimizing the impact of patient disease, increasing attention has been given to the two other components of perioperative risk: surgery and anesthesia. Because of the lower probability of death associated with these two factors, a different methodologic approach is often needed. Single-site studies are frequently not feasible, and assessments of cause of death by individual clinicians may not be appropriate. Because death is less common, outcomes other than death may be studied to obtain a statistically reliable number of adverse events. This approach has particularly been used with respect to the development of clinical predictors of cardiac events, including nonfatal myocardial infarction. 7,11 
Although investigators have continued to rely on multicenter studies, the problem has increasingly been studied through the use of administrative data sets. 12 Examples of administrative databases include Medicare claims files, private insurance company claims, and hospital electronic records. These databases have a limited amount of data for an extremely large number of subjects. For example, the Medicare database includes both financial data and ICD-9 (disease) and CPT (procedure) codes for each patient. They also include information regarding location of care and provider.
Administrative data sets have been used to derive risk-adjusted provider specific mortality. For example, New York and Pennsylvania have used this type of data to develop a system of individual physician report cards for cardiac surgery, which allow consumers to determine their surgeon's risk-adjusted mortality rate. 13,14 
In studying the influence of anesthesia-related factors, previous studies have suggested that its contribution is several orders of magnitude lower than the influence of patient disease or surgical error. During the previous two decades, there have been attempts to quantify the risk associated with anesthesia using a variety of sources. Slogoff and Keats 15 reported on rates of perioperative myocardial ischemia and infarction after cardiac surgery and found that one anesthesia provider (Anesthesiologist 7) had a significantly higher rate than the others. The Confidential Enquiry into Perioperative Deaths assessed nearly 1 million cases of anesthesia during a 1-yr period in 1987 in three large regions of the United Kingdom. 5 There were 4,034 deaths within 30 days in an estimated 485,850 operations, resulting in a crude mortality rate of 0.7–0.8%. Surgery had contributed totally or partially in 30% of all patients. Progression of the presenting disease had contributed to death in 67.5% of all patients, with progress of an intercurrent disease being relevant in 44.3% of patients. Anesthesia was considered to be the sole cause of death in only 3 individuals, for a rate of 1 in 185,000 cases, and anesthesia was contributory in 410 deaths, for a rate of 7 in 10,000. Therefore, the ability to determine the influence of the anesthesia provider on perioperative mortality would require either an extremely large sample size or alternative outcome measures. Silber et al.  have used large data sets and have developed additive outcome measures in the study of the factors associated with anesthesia. 12,16 
In this issue of Anesthesiology, Silber et al.  1 have continued their inquiries into hospital and provider characteristics for perioperative mortality by evaluating death and failure to risk in a large cohort of Medicare patients. Silber et al.  developed the concept of failure to rescue as a means of studying quality of care for the manner in which hospitals handle 41 known “emergencies” or complications that are coded in the Medicare files. 17,18 In brief, they evaluated the 30-day death rate in patients who died without a recorded complication or in whom a complication developed. The theoretical foundation for such an approach is that the presence of more “skilled” care would prevent complications from becoming deaths. By evaluating both mortality and failure to rescue, the number of relevant outcomes dramatically increases, allowing investigators to study the influence of care providers on outcome on a smaller number of cases.
In a previous issue of Anesthesiology, Silber et al.  16 compared anesthesia care personally performed or medically directed by an anesthesiologist with the outcomes of patients whose anesthesia was not personally performed or medically directed by an anesthesiologist and reported that both 30-day mortality rate and failure to rescue were lower when anesthesiologists directed anesthesia care. Work by this group and others has shown that hospitals with higher percentages of board-certified anesthesiologists on staff had lower adjusted mortality and failure-to-rescue rates among surgical patients. 17,19 The current study further evaluates the importance of board certification by specifically comparing outcomes in anesthesiologists who were board certified with those who were not.
Although this article represents a significant contribution to the literature, any clinical or policy recommendations to be derived from this article must recognize the limitations imposed by the data set and this type of analysis. Clinicians need to know that this type of analysis is designed to generate hypotheses, not to test hypotheses. Because all the relevant data are not available from Medicare claims data, it is not possible to reach definitive conclusions about the causes of death or the other outcomes using this analytic approach.
First, it is not clear from the data whether the actions of the anesthesiologist contributed to the patient's death. As noted in the article, the estimated number of excess deaths derived in this article—3.8 excess deaths per 1,000 cases—is several magnitudes greater than the often-quoted risk of death from anesthesia of 0.005 deaths per 1,000 cases. This is in the range of mortality associated with the other components of perioperative risk: surgery and patient disease. The data do not permit the authors to link specific actions of the anesthesiologist to the death rate, and therefore, there is no direct evidence that all of the excess deaths can be attributed to whether the anesthesiologist had a board certification. However, extrapolating from the foundations of the failure-to-rescue concept, more skilled providers of care (e.g.  , board certification as a surrogate for “skill”) may reduce the severity and fatality of a complication, which may not be directly related to anesthesia care. This may be related to such interventions as more skilled resuscitation or better pharmacologic management of coexisting disease. Information not available in the claims data would be required to demonstrate this link.
Second, the article attributes the difference in death rates to board certification based on a set of odds ratios. In statistics, omitted variable bias occurs when variables not in the equation are correlated with a variable that is already in the equation. Board certification is included in the equation; however, many variables that are probably correlated with board certification are not included in the equation. These include training at better medical schools and residency programs, being educated at US medical schools, and other factors. These variables, not board certification, could be responsible for the higher death rate. Statistically, the greater the correlation between the omitted variables and the variable included in the equation (board certification), the greater the omitted variable bias. That is, it is possible that some other factor highly correlated with board certification (i.e.  , an omitted variable) is actually responsible for the higher death rate.
This hypothesis of some other factor being responsible is given some legitimacy in figure 1. The odds ratio seems to be meaningfully different from 1.0 only in the 16- to 20-yr postgraduation cohort. In the other years, the ratio is not significantly different from 1.0. Therefore, it does not seem to be the fact that the physician passed the Boards that is important, but that they passed the Boards 16–20 yr ago. Further analysis of what those individuals 16–20 yr after graduation had in common is warranted to determine what denotes a more “skilled” clinician and whether it is the certification process itself or the factors leading to board certification.
Third, there were only 75 anesthesiologists who were not board certified 11–25 yr after medical school graduation. This is a very small number of anesthesiologist to draw definitive clinical or policy conclusions. Because these 75 anesthesiologists can then be subdivided into US-trained and foreign-trained medical graduates, are concentrated in relatively few hospitals, and probably are located in relatively few geographic areas, the ability to generalize the results becomes even more problematic.
Despite these limitations, analysis of Medicare claims offers a unique opportunity to generate hypotheses that warrant further study. The previous study by Silber et al.  16 regarding medical direction appropriately has led to calls for further research about the relative risks associated with anesthesiologists and nurse anesthetists. Similarly, the current study should stimulate further questions with regard to the importance of board certification with respect to patient outcomes.
The specialty of anesthesiology has been lauded for its early adoption of systems to improve patient safety and for being at the forefront of the six sigma process. 20 These processes have led to rare fatalities directly related to the provision of anesthesia care. The exploration by Silber et al.  of factors associated with reduced rates of failure to rescue provides one innovative way of further reducing the elements of perioperative risk. Board certification of the anesthesiologist, or some unidentified aspect of the educational or certification process, warrants further study as a means of reducing the mortality associated with perioperative complications related to anesthesia, surgery, or patient disease. Because the risk directly related to anesthesia seems to be almost too low to measure easily, the goal should be to identify additional actions or training within the domain of the anesthesiologist that ensures that our patients receive the best quality of care. The specialty should focus on determining the link between provider characteristics and failure to rescue and designing interventions to further reduce all-cause perioperative mortality.
Silber JH, Kennedy SK, Even-Shoshan O, Chen W, Mosher RE, Showan AM, Longnecker DE: Anesthesiologist board certification and patient outcomes. A nesthesiology 2002; 96: 1044–52Silber, JH Kennedy, SK Even-Shoshan, O Chen, W Mosher, RE Showan, AM Longnecker, DE
Ruth HS: Anesthesia study commissions. JAMA 1945; 127: 514–7Ruth, HS
Beecher HK, Todd DP: A study of deaths associated with anesthesia and surgery. Ann Surg 1954; 140: 2–34Beecher, HK Todd, DP
Lunn JN, Hunter AR, Scott DB: Anaesthesia-related surgical mortality. Anaesthesia 1983; 38: 1090–6Lunn, JN Hunter, AR Scott, DB
Lunn JN, Devlin HB: Lessons from the confidential enquiry into perioperative deaths in three NHS regions. Lancet 1987; 2: 1384–6Lunn, JN Devlin, HB
Tiret L, Desmonts JM, Hatton F, Vourc'h G: Complications associated with anaesthesia: A prospective survey in France. Can Anaesth Soc 1986; 33: 336–44Tiret, L Desmonts, JM Hatton, F Vourc'h, G
Goldman L, Caldera DL, Nussbaum SR, Southwick FS, Krogstad D, Murray B, Burke DS, O'Malley TA, Goroll AH, Caplan CH, Nolan J, Carabello B, Slater EE: Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med 1977; 297: 845–50Goldman, L Caldera, DL Nussbaum, SR Southwick, FS Krogstad, D Murray, B Burke, DS O'Malley, TA Goroll, AH Caplan, CH Nolan, J Carabello, B Slater, EE
Mangano DT, Layug EL, Wallace A, Tateo I: Effect of atenolol on mortality and cardiovascular morbidity after noncardiac surgery. Multicenter Study of Perioperative Ischemia Research Group. N Engl J Med 1996; 335: 1713–20Mangano, DT Layug, EL Wallace, A Tateo, I
Poldermans D, Boersma E, Bax JJ, Thomson IR, van de Ven LL, Blankensteijn JD, Baars HF, Yo TI, Trocino G, Vigna C, Roelandt JR, van Urk H: The effect of bisoprolol on perioperative mortality and myocardial infarction in high-risk patients undergoing vascular surgery. Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group. N Engl J Med 1999; 341: 1789–94Poldermans, D Boersma, E Bax, JJ Thomson, IR van de Ven, LL Blankensteijn, JD Baars, HF Yo, TI Trocino, G Vigna, C Roelandt, JR van Urk, H
Rose DK, Cohen MM, DeBoer DP: Cardiovascular events in the postanesthesia care unit: Contribution of risk factors. A nesthesiology 1996; 84: 772–81Rose, DK Cohen, MM DeBoer, DP
Eagle K, Brundage B, Chaitman B, Ewy G, Fleisher LA, Hertzer N, Leppo L, Ryan T, Schlant R, Spencer W, Spittell J, Twiss R: Guidelines for perioperative cardiovascular evaluation of the noncardiac surgery. A report of the American Heart Association/American College of Cardiology Task Force on Assessment of Diagnostic and Therapeutic Cardiovascular Procedures. Circulation 1996; 93: 1278–317Eagle, K Brundage, B Chaitman, B Ewy, G Fleisher, LA Hertzer, N Leppo, L Ryan, T Schlant, R Spencer, W Spittell, J Twiss, R
Cohen M, Duncan PG, Tate RB: Does anesthesia contribute to operative mortality? JAMA 1988; 260: 2859–63Cohen, M Duncan, PG Tate, RB
Hannan EL, Kumar D, Racz M, Siu AL, Chassin MR: New York State's Cardiac Surgery Reporting System: Four years later. Ann Thorac Surg 1994; 58: 1852–7Hannan, EL Kumar, D Racz, M Siu, AL Chassin, MR
Schneider EC, Epstein AM: Use of public performance reports: A survey of patients undergoing cardiac surgery. JAMA 1998; 279: 1638–42Schneider, EC Epstein, AM
Slogoff S, Keats AS: Does perioperative myocardial ischemia lead to postoperative myocardial infarction? A nesthesiology 1985; 62: 107–14Slogoff, S Keats, AS
Silber JH, Kennedy SK, Even-Shoshan O, Chen W, Koziol LF, Showan AM, Longnecker DE: Anesthesiologist direction and patient outcomes. A nesthesiology 2000; 93: 152–63Silber, JH Kennedy, SK Even-Shoshan, O Chen, W Koziol, LF Showan, AM Longnecker, DE
Silber JH, Williams SV, Krakauer H, Schwartz JS: Hospital and patient characteristics associated with death after surgery: A study of adverse occurrence and failure to rescue. Med Care 1992; 30: 615–29Silber, JH Williams, SV Krakauer, H Schwartz, JS
Silber JH, Rosenbaum PR: Measuring quality of hospital care. JAMA 1995; 273: 21Silber, JH Rosenbaum, PR
Hartz AJ, Krakauer H, Kuhn EM, Young M, Jacobsen SJ, Gay G, Muenz L, Katzoff M, Bailey RC, Rimm AA: Hospital characteristics and mortality rates. N Engl J Med 1989; 321: 1720–5Hartz, AJ Krakauer, H Kuhn, EM Young, M Jacobsen, SJ Gay, G Muenz, L Katzoff, M Bailey, RC Rimm, AA
Kohn LT, Corrigan JM, Donaldson MS: To Err Is Human: Building a Safer Health System. Washington DC, Institute of Medicine, 2000