Free
Perioperative Medicine  |   July 2020
Incidence and Classification of Nonroutine Events during Anesthesia Care
Author Notes
  • From the Geriatric Research Education and Clinical Center, Tennessee Valley Veterans Affairs Healthcare System, Nashville (J.S.L., J.M.S., M.B.W.); the Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center (J.S.L., J.M.S., G.W., M.S.S., E.P., M.B.W.); and the Departments of Anesthesiology (all authors); Biomedical Informatics (M.B.W.), and Biostatistics (M.S.S.), Vanderbilt University School of Medicine and Vanderbilt University Medical Center, Nashville, Tennessee.
  • This article is featured in “This Month in Anesthesiology,” page 1A.
    This article is featured in “This Month in Anesthesiology,” page 1A.×
  • This article is accompanied by an editorial on p. 8.
    This article is accompanied by an editorial on p. 8.×
  • Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal’s Web site (www.anesthesiology.org).
    Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal’s Web site (www.anesthesiology.org).×
  • This article has a visual abstract available in the online version.
    This article has a visual abstract available in the online version.×
  • Early incomplete aspects of this work were presented at the 2002 meeting of the Association of University Anesthetists on April 12, 2002, in Nashville, Tennessee, and the 2004 International Anesthesia Research Society annual meeting on March 27, 2004, in Tampa, Florida.
    Early incomplete aspects of this work were presented at the 2002 meeting of the Association of University Anesthetists on April 12, 2002, in Nashville, Tennessee, and the 2004 International Anesthesia Research Society annual meeting on March 27, 2004, in Tampa, Florida.×
  • Submitted for publication December 27, 2018. Accepted for publication March 31, 2020. Published online first on May 8, 2020.
    Submitted for publication December 27, 2018. Accepted for publication March 31, 2020. Published online first on May 8, 2020.×
  • Address correspondence to Dr. Liberman: Center for Research and Innovation in Systems Safety, 1211 21st Avenue South, Medical Arts Building, Suite 732, Nashville, Tennessee 37212. justin.s.liberman@vumc.org. This article may be accessed for personal use at no charge through the Journal Web site, www.anesthesiology.org.
Article Information
Perioperative Medicine / Clinical Science / Airway Management / Cardiovascular Anesthesia / Education / CPD / Ethics / Medicolegal Issues / Patient Safety / Technology / Equipment / Monitoring / Quality Improvement
Perioperative Medicine   |   July 2020
Incidence and Classification of Nonroutine Events during Anesthesia Care
Anesthesiology 7 2020, Vol.133, 41-52. doi:https://doi.org/10.1097/ALN.0000000000003336
Anesthesiology 7 2020, Vol.133, 41-52. doi:https://doi.org/10.1097/ALN.0000000000003336
Abstract

Background: A nonroutine event is any aspect of clinical care perceived by clinicians or trained observers as a deviation from optimal care based on the context of the clinical situation. The authors sought to delineate the incidence and nature of intraoperative nonroutine events during anesthesia care.

Methods: The authors prospectively collected audio, video, and relevant clinical information on 556 cases at three academic hospitals from 1998 to 2004. In addition to direct observation, anesthesia providers were surveyed for nonroutine event occurrence and details at the end of each study case. For the 511 cases with reviewable video, 400 cases had no reported nonroutine events and 111 cases had at least one nonroutine event reported. Each nonroutine event was analyzed by trained anesthesiologists. Rater reliability assessment, comparisons (nonroutine event vs. no event) of patient and case variables were performed.

Results: Of 511 cases, 111 (21.7%) contained 173 nonroutine events; 35.1% of event-containing cases had more than one nonroutine event. Of the 173 events, 69.4% were rated as having patient impact and 12.7% involved patient injury. Longer case duration (25th vs. 75th percentile; odds ratio, 1.83; 95% CI, 1.15 to 2.93; P = 0.032) and presence of a comorbid diagnosis (odds ratio, 2.14; 95% CI, 1.35 to 3.40; P = 0.001) were associated with nonroutine events. Common contributory factors were related to the patient (63.6% [110 of 173]) and anesthesia provider (59.0% [102 of 173]) categories. The most common patient impact events involved the cardiovascular system (37.4% [64 of 171]), airway (33.3% [57 of 171]), and human factors, drugs, or equipment (31.0% [53 of 171]).

Conclusions: This study describes characteristics of intraoperative nonroutine events in a cohort of cases at three academic hospitals. Nonroutine event–containing cases were commonly associated with patient impact and injury. Thus, nonroutine event monitoring in conjunction with traditional error reporting may enhance our understanding of potential intraoperative failure modes to guide prospective safety interventions.

Editor’s Perspective:

What We Already Know about This Topic:

  • A nonroutine event is defined as any aspect of clinical care perceived by clinicians or observers as a deviation from optimal care for a patient in a clinical situation

  • Nonroutine events are frequent and associated with increased clinician workload and patient physiologic disturbances

What This Article Tells Us That Is New:

  • Video recordings of 511 cases from 1998 to 2004 were viewed to identify nonroutine events, which occurred in 22% of cases, and some cases had multiple events

  • One in fifteen patients had events associated with some degree of patient injury

  • The most common contributory factors were related to provider, patient, or teaching/supervision

Substantial progress has been made in describing the prevalence of preventable medical error and its contribution to degraded clinical quality and patient outcomes.1  The most common method of capturing patient safety data is event reporting2  which typically focuses on serious adverse or “never” events.3  Providers are often reluctant to report errors for fear of social, legal, and regulatory retribution.4,5  Clinical practice frequently deviates from what would be considered “optimal” care,3,6,7  but events that do not cause substantial patient harm are rarely reported.8,9 Post hoc event analyses are inherently biased and of variable value in improving systems.2  Thus, current event reporting systems do not reliably identify systemic factors (e.g., technology or process design issues, conflicting priorities) which often contribute to adverse events and understanding their role is essential to designing safety mitigations.
Fifteen years ago, we introduced the concept of the nonroutine event (called an NRE in other papers) to health care, 2,3,10  modeled after safety processes in the nuclear power industry where every deviation from standard operating procedures is reported and investigated. Unfortunately, in most areas of health care, we typically do not have standard operating procedures. Moreover, patients present to us in less than perfect and often ill-described condition. Thus, in perioperative care, there may be appreciable variability in the “optimal care path” for a given patient when compared to others presenting for similar procedures depending on each patient’s unique characteristics and needs.
A nonroutine event is defined as any aspect of clinical care perceived by clinicians or observers as a deviation from optimal care for that patient in that clinical situation. The nonroutine event concept extends what is considered valuable safety information beyond the occurrence or near-occurrence of patient injury. While nonroutine events include “near-misses” and “critical incidents,” other nonroutine events elucidate flawed care processes (e.g., missing or broken equipment, delayed lab tests, interpersonal communication failures, etc.) that may not be immediately or directly linked with actual or potential patient injury and therefore are unreliably detected by current event reporting systems.7 
In previous studies analyzing thousands of cases using the Comprehensive Open-ended Nonroutine Event Survey,7  we found that nonroutine events: (1) were frequent (greater than or equal to one in 20 to 40% of all care periods studied); (2) capture a wide cross-section of system failures; and (3) were associated with increased clinician workload and significant patient physiologic disturbances.2,3,7,10–13  We now report on a large corpus of video-recorded intraoperative cases to delineate in detail the incidence, severity, and contributors of anesthesia nonroutine events.
Materials and Methods
In this prospective observational study, we video recorded anesthesia cases, captured data about the provider, patient, and case, measured clinician workload, and used the nonroutine event survey tool7  (completed in the postanesthesia care unit by the in-room anesthesia provider) to identify cases containing nonroutine events. Domain experts then reviewed the videos and accompanying data to understand the epidemiology of these events. Anesthesia was always conducted at the providers’ discretion according to local clinical practices.
Patient and Case Selection
On the evening before surgery, a trained research assistant reviewed the next day’s clinical schedule and identified the most appropriate cases to achieve case diversity, focusing on general anesthesia cases, and to avoid participant fatigue due to repeated observations. Cardiac surgery cases were excluded due to case complexity. The morning of surgery, the research assistant reviewed potential cases and then sequentially sought patient, as well as primary anesthesia provider (anesthesia resident or Certified Registered Nurse Anesthetist), approval. Written, informed consent was obtained from both the primary anesthesia provider and attending. The patient provided verbal consent to allow the study of their anesthesia provider and signed both a written video release and a Health Insurance Portability and Accountability Act waiver. Verbal assent was obtained from all other members of the operating room team. At any time, any operating room staff member could ask that video recording cease. The study protocol and consent process were approved by all three hospitals’ Human Subjects Protection Programs (San Diego, California). Note that the study site that accounted for the preponderance of cases was a Veterans Affairs healthcare facility, and, because of this, we did not include obstetric or pediatric cases in our dataset.
Data Collection
We prospectively collected audio/video and relevant clinical information (e.g., all patient vital signs) from the time the patient entered the operating room until the patient was transported out of the operating room. A trained research assistant remained in the operating room to operate the audio–visual equipment while concurrently collecting case-specific event, time-motion, workload, and vigilance data.10,14  Custom hardware and software tools facilitated data collection and analysis. Both the primary anesthesia provider (resident or nurse anesthetist) and the attending anesthesiologist wore wireless unidirectional lavaliere microphones. Data were collected prospectively in real-time and encoded with a time code signal allowing synchronization of multiple data feeds. The data were collated and archived to digital versatile disc. All identifiable patient and provider information were excised (other than the images and voices on the videos). Data were managed with secure processes approved by the Institutional Review Board (San Diego, California) and all hospitals’ risk management leadership.10 
Other Provider and Case Metrics
Baseline Data Collection.
Anesthesia provider demographic data were obtained via a brief questionnaire.
Real-Time Workload Measurements
Psychologic workload was assessed by the trained observer and by the participants themselves using a Borg Workload Scale.10,14,15,16  This visual analog scale, ranging from 6 (no exertion) to 20 (maximum exertion) yields continuous parametric data.14,16  At random 8- to 12-min intervals, the computer prompted the observer to score the participant’s workload and then query the clinician for his/her own workload rating. Clinician and observer workload ratings typically correlate (e.g., R ≥ 0.7).10,14,16  While this technique may not accurately reflect rapidly changing workload, it has been successfully used in several studies of performance shaping factors in anesthesia.10,12,14,16–18 
A procedural workload measure, the response latency to an “alarm light,” was assessed.10,14,16  A bright red 1-cm light, placed next to the physiologic monitor in the anesthesia monitoring array, was illuminated at 7- to 11-min random intervals. Participants were instructed to respond with either a verbal or manual indication as soon as they detected illumination of the light. Previous studies have shown response latency to be affected by differing levels of training,16  the introduction of technology,14  and of intraoperative teaching.10 
Nonroutine Event Data Capture
The Comprehensive Open-Ended Nonroutine Event Survey (called CONES in other papers)7  was utilized by trained nonclinician observers to elicit possible nonroutine events based on clinicians’ responses to nine yes/no questions. Upon identification of a nonroutine event, open-ended questions were used to solicit information regarding potential contributing factors and etiology.
Event Database Entry, Classification, and Review
Each case studied was assigned a random case number to assure anonymity. All participant and patient identifiers were removed. Standardized forms were used to capture the relevant case summary data elements. A custom case database, written in open-source software, included all of the study variables. A comprehensive data dictionary specified for each data element the type of variable (e.g., categorical, numeric), units, range, and missing data code. All case data elements were double-checked for consistency, completeness, and plausibility.
Video Review Process
Video raters went through a rigorous training process on the definition and verification of nonroutine events, as well as the application of the patient injury, contributory factor, and patient impact event coding schemes. The trained experts, who were all board-certified anesthesiologists, independently reviewed events visible on video and followed the structured coding process. Each nonroutine event was assigned a primary reviewer. For the purposes of intra- and interrater reliability analysis, some nonroutine events were assigned to multiple raters, or to the same rater more than once. In a random subset of events where there was disagreement between reviewers, they met with each other or with Dr. Weinger to achieve consensus and refine the coding scheme as necessary.
Each nonroutine event was categorized according to its patient impact, patient injury type, outcome severity, and putative contributing factors. Patient injury was defined as any unanticipated side effect or complication of anesthesia that affected the patient’s clinical course or quality of life. With this patient-centric definition, “injuries” included events that required the need for a higher level or prolonged care (e.g., Intensive Care Unit, admission instead of discharge home after surgery) or emotional distress (e.g., elective case cancellation due to nonpatient factors). A structured taxonomy of 337 patient impact events, organized hierarchically by clinical manifestation,7  was applied (Supplemental Digital Content table 1, http://links.lww.com/ALN/C361). This event taxonomy had been developed and refined previously, informed by taxonomic research of anesthesia events.12,19–24  Whenever possible, reviewers coded the patient impact events in the chronological order in which they occurred to provide a structured description of event progression. Not all nonroutine events are associated with patient impact events.
Reviewers also assessed whether any of the 21 discrete Contributory Factors appeared to play a role in the occurrence of each nonroutine event. The Contributory Factor analysis was intended to delineate differences between types of events and potential common sources or mechanisms of event etiology. The previously used Contributory Factor classification was based on the literature and 20 yr of human factors experience.2,7  Note that the Patient Impact Event and Contributory Factor coding was always done in that order and that there was some intentional overlap in the nature of the codes (e.g., Patient Impact Event category 930: Equipment Problems and Contributory Factor category—Equipment Factors: Equipment Failure) since the combination of the two coding schemes were used primarily for explanatory rather than taxonomic purposes. Finally, the raters were asked to estimate the likelihood of event reoccurrence (1 = improbable to 5 = frequent) and the average event severity if the event were it to recur on numerous occasions (1 = negligible and 5 = catastrophic).
Statistical Analysis
Previous work allowed us to anticipate an overall incidence of anesthesia nonroutine events of 30 to 35%; no statistical power calculation was conducted before the study.7  The Comprehensive Open-ended Nonroutine Event Survey responses were used to categorize cases as either not containing any nonroutine events (case proceeded per “routine”) or containing nonroutine events. Only event-containing cases with video files that could be reviewed and coded were included in the analyses (fig. 1). For this analysis, nonroutine event ratings from the primary clinician reviewer were used. To assess the intra- and interrater reliability of our expert raters, we compared pairs of ratings (either by two raters or by the same rater twice) and calculated the overall percent positive and negative agreement, and Cohen κ coefficients
Fig. 1.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
Fig. 1.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
×
Univariate comparisons in patient, case, and provider attributes between event-containing and routine (no events reported) cases were performed using Wilcoxon (for numeric data), Pearson chi-square (for categorical data), or Fisher exact test (for categorical data where any expected cell count was less than 5). A two-sided P value of 0.05 was chosen as the statistical criteria of interest. All subgroup analyses were preplanned.
Multiple logistic regression was used to evaluate the independent associations between key perioperative factors and the odds of nonroutine event occurrence. A logistic regression was performed for event occurrence with case type as a function of preplanned predictors case duration, hospital, patient age, patient sex, and American Society of Anesthesiologists (ASA; Schaumburg, Illinois) Physical Status with restricted cubic splines with three knots on case duration and patient age. Cubic splines are piecewise functions that help the model to accommodate nonlinear variables. A sandwich estimator adjusted for the correlation introduced as a result of repeated cases on provider identification. A multiple degrees of freedom Wald test was used to test all effects. The interaction of patient age and patient sex were not significant, so we refit the models with this interaction removed.
All statistical analyses were implemented using R version 3.5.1 (R Core Team, 2014).
Results
Of the 556 eligible study cases with video files, 16 case videos were not reviewable due to technical difficulties (e.g., no sound) and in 29 cases, the reported nonroutine events were not visible on the video recording (fig. 1). Of the remaining 511 cases, 173 events were reported in 111 cases (21.7% [111 of 511]). Among these nonroutine event–containing cases, 39 cases (35.1% [39 of 111]) contained more than one event; 23 cases (20.7% [23 of 111]) contained two reported nonroutine events and 16 cases (14.4% [16 of 111]) had three or more events.
Patient and anesthesia provider demographic variables (table 1) significantly associated with the report of a nonroutine event in univariate analyses included: patient age (e.g., patients were more likely to be older in event-containing cases), ASA Physical Status, and provider type (e.g., residents earlier in their training were more likely to report nonroutine events). Case-related variables (table 2) associated with events included hospital site, type of anesthetic, type of surgery, and case duration (e.g., longer cases were more likely to contain events; table 2).
Table 1.
Demographic Variables
Demographic Variables×
Demographic Variables
Table 1.
Demographic Variables
Demographic Variables×
×
Table 2.
Study Case and Surgical Variables
Study Case and Surgical Variables×
Study Case and Surgical Variables
Table 2.
Study Case and Surgical Variables
Study Case and Surgical Variables×
×
Rater Reliability
A random sample of 15 nonroutine events was selected for repeat assessment of contributory factors coding by the primary rater (G.W.). Intrarater reliability was assessed using these 15 pairs of contributory factors ratings and was very good (Supplemental Digital Content table 2, http://links.lww.com/ALN/C361). For example, for contributory factors, the overall percent agreement ranged from 80 to 100% with positive percent agreement (i.e., present in both rating instantiations) better (86 to 100%) than negative percent agreement (50 to 100%). Cohen κ for contributory factor coding ranged from 0.44 to 1.00. The lower scores were due to low negative percent agreement in the team and technology categories.
Interrater reliability was assessed using 113 pairs of patient impact events ratings for the primary reviewer and one secondary reviewer and was overall very good to excellent. For example, the overall percent agreement ranged from 82 to 95% with negative (81 to 99%) better than positive (67 to 100%) percent agreement. Cohen κ coefficients ranged from 0.42 to 0.85, with lower values in part due to the large number of available discrete coding categories and potential overlap in some categories (e.g., pulmonary vs. respiratory events) of the Patient Impact Event taxonomy.
Expert Ratings
A majority of nonroutine events (69.4% [120 of 173]; Supplemental Digital Content table 3, http://links.lww.com/ALN/C361) were rated as having patient impact while 12.7% [22 of 173] were rated as involving patient injury. Near misses were observed in 84 of the nonroutine events (48.6% [84 of 173]). Raters’ assessment of the likelihood of nonroutine event recurrence had a median value of 3.0 (interquartile range, 2.0 to 3.0; Supplemental Digital Content table 3, http://links.lww.com/ALN/C361) with a range of 1 (improbable) to 5 (frequent). The predicted event severity if the nonroutine event recurred was assigned a median rating of 2.0 (interquartile range, 2.0 to 3.0) with a range of 1 to 4.5 (1 = negligible and 5 = catastrophic). Providers in the study could be observed multiple times. The median number of times observed was 3 (interquartile range, 1 to 7). The minimum and maximum number of times observed was 1 and 46, respectively.
Patient Impact Events
Ninety-nine percent (171 of 173) of nonroutine events were coded as containing at least one Patient Impact Event. The most common Patient Impact Event categories (represented as percent of events with one or more rating [n = 171]; Supplemental Digital Content table 1, http://links.lww.com/ALN/C361) were the cardiovascular system (37.4% [64 of 171]), airway (33.3% [57 of 171]), and human factors, drugs, and equipment (31.0% [53 of 171]). A majority of Airway Associated Events were related to difficulty with tracheal intubation (14.6% [25 of 171]), esophageal intubation (7.6% [13 of 171]), or premature extubation (5.8% [10 of 171]). An example of an airway Patient Impact Event (event #11,243) involved an inexperienced provider starting a case without the attending present for induction, the provider was unable to intubate the patient despite three attempts at laryngoscopy. The attending arrived and addressed the suboptimal intubating conditions (patient positioning) and the resident then successfully intubated the patient on the next attempt at direct laryngoscopy.
The majority of patient impact events falling in the cardiovascular category were related to unstable hemodynamics (i.e., significant hypotension or hypertension; 32.2% [55 of 171]), but there were many different causes. In one case (event #11,086), progressive intraoperative ST segment elevation was associated with significant hypotension and modest tachycardia. All three of this patient’s new coronary bypass grafts had occluded immediately following protamine administration. The patient impact event codes for this case were myocardial ischemia (electrocardiogram evident; code 213) and hypotension—controllable (code 263).
The Patient Impact Event category of human factors, drugs, and equipment (31.0% [53 of 171] of all nonroutine events; Supplemental Digital Content table 1, http://links.lww.com/ALN/C361) includes not only drug errors (12.9% [22 of 171]), but also equipment problems (15.2% [26 of 171]), and one event associated with clinical staff injury during intraoperative care. Some of the patient impact events in this category describe deficient processes rather than patient’s clinical findings or outcomes. For example, human factors, drugs, and equipment code #934, “Use Error,” was assigned by reviewers to an event involving the failure to secure arterial line and its subsequent unintended removal due to patient positioning. Errors in drug administration accounted for 12.9% (22 of 171) of all patient impact events and included drug overdose (4.7% [8 of 171]), drug underdose (4.1% [7 of 171]), opioid overdose (1.8% [3 of 171]), and the need for emergency rescue medications (i.e., naloxone or flumazenil; 1.2% [2 of 171]). Our Patient Impact Event taxonomy captured issues with equipment (15.2% [26 of 171]), which included anesthesia equipment failure or defect (7.0% [12 of 171]), disconnect from device (i.e., intravenous catheter, wires; 2.5% [6 of 171]), problem with in-dwelling device (pacemaker, pulmonary artery catheter; 1.2% [2 of 171]), and a use error related to incorrect programing of an intravenous pump (2.3% [4 of 171]).
Contributory Factors
Nonroutine event–containing cases had from 1 to 12 Contributory Factors with a median of 4 (interquartile range, 2.5 to 6.0). The most common contributor categories were related to patient factors (63.6% [110 of 173]) and the in-room provider (59.0% [102 of 173]; table 3). Preexisting patient disease and unexpected physiologic response to routine interventions were the two most frequently chosen individual contributory factors. Examples include a patient with morbid obesity who had difficult mask ventilation (preexisting disease) and another patient with severe hypoventilation after a small dose bolus of fentanyl (unexpected response). Figure 2 and Supplemental Digital Content table 4 (http://links.lww.com/ALN/C361) provides complete descriptions of selected nonroutine events and their associated contributory factors and patient impact events.
Fig. 2.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
Fig. 2.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
×
Table 3.
Contributory Factors in Nonroutine Event–Containing Cases
Contributory Factors in Nonroutine Event–Containing Cases×
Contributory Factors in Nonroutine Event–Containing Cases
Table 3.
Contributory Factors in Nonroutine Event–Containing Cases
Contributory Factors in Nonroutine Event–Containing Cases×
×
When the in-room clinician was a trainee, it was common for reported events to be coded with the provider-related factor “inexperience” (33.5% of all events [58 of 173]; table 3). Other common provider factors were “inadequate knowledge” (20.2% [35 of 173]) and “error in judgment” (18.5% [32 of 173]). For example, an esophageal intubation by a medical student was coded as “provider inexperience.” A neurosurgical patient who moved upon insertion of head pins despite a direct query by the surgeon as to whether the patient was appropriately anesthetized was coded as both error in judgment and inexperience. About a third of all event-containing cases contained teaching-related factors, for example, a case of laryngospasm when a resident extubated a patient prematurely in the absence of the attending (coded as inadequate supervision). Interruption or distraction was a factor in 14.5% (25 of 173) of all events.
Nonanesthesia members of the perioperative team contributed to events in more than one-third of events. Surgeons’ actions (or inactions) or other aspects of the surgical procedure were deemed contributory in 24.3% (42 of 173) of events. In one case, the patient became hypertensive shortly after a surgeon requested to deepen anesthesia just as the anesthesia resident was preparing to extubate the patient. In another case, surgery was cancelled after induction of anesthesia because the attending surgeon was unavailable despite preinduction assurances to the contrary (coded with both team/communication and surgeon-related contributory factors).
In 19.1% (33 of 173) of all event-containing cases, problems with technology were deemed contributory. Examples included an unintended 10-fold overdose of opioid infusing through an infusion pump due to a programming error resulting in significant hypotension (equipment usability) and an inability to remove a piece of surgical equipment which necessitated waking the patient up in the lateral position and resulted in hypoxia on extubation (equipment failure).
Provider Workload and Vigilance
Subjective workload and response times to the vigilance light (i.e., 20- to 26-s median values across all three phases for all cases) were similar to that seen in previous studies.7,16,23–25  Neither observer-reported workload nor participant-reported workload, nor response to the vigilance light were significantly different in event-containing versus routine cases (Supplemental Digital Content table 5, http://links.lww.com/ALN/C361).
Logistic Regression Analysis
In the regression model, after adjusting for other factors, there was no evidence of any preinduction patient or provider factor being associated with a higher incidence of event reporting. Nonroutine events were more likely to be reported at the Veterans Affairs hospital site versus the other two (A vs. B odds ratio, 6.04; 95% CI, 0.74 to 49.36; A vs. C odds ratio, 7.76; 95% CI, 0.96 to 62.54; P = 0.048). Neither patient age nor ASA Physical Status was associated with event reporting. Longer case duration was associated with an increased incidence of nonroutine events (odds ratio, 1.81; 95% CI, 1.14 to 2.86; P = 0.030; comparing 25th vs. 75th percentile of case duration). A sensitivity analysis using data only from the largest hospital was consistent with these findings. Multivariable analysis utilizing comorbidity data was conducted as a post hoc exploratory analysis and found that the presence of a comorbid diagnosis was associated with the presence of an event (odds ratio, 2.14; 95% CI, 1.35 to 3.40; P = 0.001).
Discussion
This study describes in detail the incidence and nature of intraoperative nonroutine events in a large prospective cohort at three academic medical centers. Almost one-quarter of study cases contained nonroutine events and more than 90% of these events were visible on video recording. Thirty-five percent of event-containing cases contained multiple nonroutine events. Domain experts determined that 85% of the events had direct patient impact. Thus, nearly one in five of our patients experienced a direct impact from an event and 1 in 15 patients had events associated with some degree of patient injury. These findings are consistent with previous studies across multiple domains and studies.7,16,23–25  The resource-intense active surveillance methods used in the present study provides a level of data accuracy and detail unobtainable with other methods including traditional event self-reporting and institutional “root cause” analyses. The methods are analogous to other direct observation strategies already employed in the public health sector25  where passive self-reporting is widely appreciated to be unreliable.26 
Relationship of Nonroutine Events to Patient Outcomes
In terms of nonroutine event incidence, the current study’s results are similar to those of a previous nonvideo study that used the Comprehensive Open-ended Nonroutine Event Survey in the postanesthesia care unit.7  Compared to traditional event reporting, events captured with this survey include more lower acuity injuries, such as case cancellation or severe postoperative pain. Analysis of the factors associated with lower acuity events helps to identify patterns of systemic deficiencies that can cause more significant patient injury. Traditional event reporting is unlikely to detect all of a care system’s potential safety weaknesses.27  Less severe events typically go unreported perhaps due to the common misconception that such events do not identify important patient safety issues.9  While analysis of “near-miss” events provides insight into the risk of future injury,4  they are also underreported in most systems.9  Near-misses are captured with this survey approach as they are by definition “nonroutine.”
Patient Impact Events
The current study captured a higher yield of patient impact events than previously reported in the literature.28,29  With voluntary reporting, only 6% of hospital-based adverse drug events detected by direct observation were reported, even in the setting of severe patient harm.30  In the current study, 12.9% (22 of 171) of nonroutine events with patient impact event ratings were deemed to be “drug related.” We also captured a higher incidence of commonly underreported event types, including drug under-doses, equipment issues (e.g., unavailability, usability, and disconnects), and surgical issues affecting anesthesia care.
Some nonroutine events contained numerous patient impact events which, when coded sequentially, provide a structured timeline of the event. For example, in one event (#11,086), the first patient impact event (not necessarily the inciting factor) was an issue of surgical decision making (845), followed by hypotension (blood pressure less than 80/40 mm Hg or less than 33%, controllable; 263), hypotension (blood pressure less than 80/40 mm Hg or less than 33%, uncontrollable; 642), myocardial ischemia (electrocardiogram evident; 213), and use error (e.g., misprogramming of intravenous pump; 934). In combination with contributory factors—in the aforementioned case, for example, patient preexisting disease, inexperience, and inadequate supervision—provides a rich structured “story” about the event. Further, analyzing commonly occurring patient impact events across many types of nonroutine events can identify “at-risk” situations. For example, analyzing event codes with “equipment problem,” we identified an issue with the use of the operating room table that could be cross-referenced to events coded with the “patient positioning” contributory factor. It turned out that operating room table usability was a significant problem and pointed to potential systemic issues with maintenance, training, and purchasing decision processes.
Contributory Factors
Understanding the contextual relationships of nonroutine events allows identification of systems level risks and thus helps to guide the design of and prioritize potential interventions to decrease patient risk. Nearly 60% of the events included provider-related contributory factors (table 3). Subcategory contributors such as error in judgment, inadequate knowledge, and interruption or distraction could be judged reliably by experts observing providers’ actual behavior. Based on a system safety framework,32  most of these will not reflect shortcomings of individual providers striving to do their best within a complex and challenging clinical context. Rather, they more likely reflect systemic deficiencies in a care system that does not provide the resources and support necessary for well-meaning clinicians to prevent and mitigate the impact of nonroutine events.31  Unless it was specifically noted in the postcase event survey by the reporting provider, without deeper knowledge of the full context of the clinical care environment and underlying systemic factors, expert raters could not code whether, for example, a medication-related event was actually due to poor labeling, absence of dosing guidance in the electronic health record, failure to provide effective training in proper use, or some other higher-level contributor. Nevertheless, patterns of sharp-end contributors provide appreciable information to an organization about potential system failure modes and opportunities for improvement. For example, a high incidence of provider-related contributors suggests the need for an organization to revisit its approaches to mitigate the potential consequences (e.g., better staffing and supervision, training, and technological support).
Limitations
The events that are reported are influenced by providers’ perceptions of what is “routine,” and are vulnerable to individual practice patterns, behaviors, and expectations. Individual clinicians show tremendous variability in reporting styles.30,39  We only analyzed events that were reported using the postcase event survey process. However, this study benefited from a trained in-operating room observer who was able to identify, and encouraged inclusion of, those events during the event survey by prompting the clinician (e.g., “What about that episode of bradycardia during surgical retraction?”). We did not review “routine” cases for events, although previous work40  suggests that the event survey methodology does not capture all possible events and that routine cases also contain nonroutine events. Thus, one must view our findings as a lower bound of event incidence for the types of cases we studied.
The analysis of reported events, whether done by the clinician themselves or by external “experts,” are retrospective in nature, and thus could be influenced by both outcome and hindsight bias.10,41  Our reviewers coded while viewing the audio–video recordings, thereby decreasing the risk of recall bias. The use of audio and video introduced new technological and human subjects’ issues that required active management. Direct observation and continuous audio–video recording could have changed participants’ behavior (the “Hawthorne Effect”), although we saw no evidence of this affecting patient care practices.
Although we sought to capture a representative sample from all three sites, logistical issues surrounding the movement of the study equipment required most observations to be completed at the Veterans Affairs hospital (Hospital A) where the patients were predominantly older, sicker males. This may limit this study’s generalizability. The observations performed at the two hospitals with less data were included to provide some case and patient diversity. All study locations utilized the same trained observer and the trainees included in our study rotated at all three study sites.
Finally, the cases studied were primarily from an academic Veterans Affairs hospital where the patients were high acuity and about half of the cases were done by trainees. Some clinical practices observed in this study may not be current anesthesia practices. However, we believe that most of the findings are consistent with or mimic current clinical practices. For example, in a few reported events, residents induced patients without the attending being present. Today, residents may still manage emergency airways outside of the operating room before an attending arrives and, even in the operating room, the resident may manage an airway while the attending is busy doing another procedure or is distracted by other in-room activities (e.g., teaching a more junior trainee). Additionally, physicians in private practice may supervise new Certified Registered Nurse Anesthetists (even at 4- or 6-to-1 ratios) in which continual presence may not be feasible.
Implications for Anesthesia
Unlike traditional error reporting, nonroutine event monitoring provides the ability to detect more common, but underreported events, and to better understand the clinical context of those events. An example of how nonroutine event analysis can provide insight into systemic deficiencies or latent failure modes33  was a near miss event (#11,898) of unexpected surgeon-induced asystole that fortunately resolved without intervention. Contributors in this event included equipment usability (distraction by the use of transesophageal echocardiography, difficulty operating a commercial prefilled syringe), processes (emergency medication not readily available), logistical/system issues (nonstandardized drug location and administration methods), and ineffective communication with the surgeon.
Understanding the clinical context of nonroutine events and their associated Contributors allows implementation of changes in care process which are applicable to a wider range of clinical situations, as identical events are unlikely to recur.34  The ability to analyze nonroutine events provides insight into systemic flaws, and the potential for harm to our patients. Most importantly, nonroutine event collection and analysis may more sensitively detect patterns of risk to future care when compared to methods that rely on self-report or analysis of adverse events.4,5,7,35,36  Our findings raise the possibility that the occurrence of nonroutine events, perhaps associated with their severity or associated patient impact events, could be an indicator or intermediate outcome measure for anesthesia-related adverse events (i.e., akin to myocardial ischemia and associated troponin elevation as a surrogate marker for myocardial infarction).37,38 
Conclusion
We describe the video analysis of 511 anesthesia cases in which 173 nonroutine events were identified. Domain expert video review was challenging but feasible. The results reveal the incidence of different types of nonroutine events, their associated patient impact, and putative contributory factors. The nonroutine event framework provides a systematic way to collect detailed information about many types of deviations from optimal care, the effects of existing quality and safety mitigations, and association with longer-term patient and organizational outcomes. While producing a large volume of auditable data, this approach facilitates a nonjudgmental approach that enhances clinician participation in quality and safety improvement. Video recording and rigorous analysis of perioperative care is an important avenue for further research and practice.
Acknowledgments
This article is dedicated to Vera Greene, who passed away prior to completion of this project but who otherwise would have been a coauthor.
The assistance of David Gonzales, M.D. (Department of Anesthesia, Veterans Affairs San Diego Healthcare System, San Diego, California), Michael Pilla, M.D. (Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee), and Miyael Syeed, B.S. (Department of Anesthesiology, University of California, San Diego, La Jolla, California) was greatly appreciated. Anesthesia professionals who assisted the authors in the early and evolving video review process included Vickie Smoot, B.C.A. (Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee), Gerry Manecke, M.D. (Department of Anesthesiology, University of California, San Diego, La Jolla, California), Ramon Sanchez, M.D. (Department of Anesthesiology, University of California, San Diego, La Jolla, California), and Michael Pilla, M.D. (Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee). The authors also acknowledge the early analytical assistance of Sonia Jain, Ph.D. (Department of Biostatistics, University of California, San Diego, La Jolla, California) and David Afshartous, Ph.D. (Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, Tennessee). Finally, the authors salute the anesthesia professionals who kindly agreed to participate in this study.
Research Support
This work was supported by the Department of Veterans Affairs Health Services Research (Washington, D.C.; IIR HR 20-066 to Dr. Weinger) and the Geriatric Research Education and Clinical Center of the Tennessee Valley Veterans Affairs Healthcare System (Nashville, Tennessee), as well as, in part, the Agency for Healthcare Research and Quality (Rockville, Maryland; AHRQ R01 HS11375 to Dr. Weinger), the Anesthesia Patient Safety Foundation (Rochester, Minnesota; grant to Dr. Slagle), and by the Department of Anesthesiology and the Institute for Medicine and Public Health of Vanderbilt University School of Medicine (Nashville, Tennessee).
Competing Interests
Dr. Weinger is a Board Examiner and Objective Standardized Clinical Examination Committee member for the American Board of Anesthesiology and has consulted for Ivenix, Inc. (North Andover, Massachusetts). Dr. Lorinc is a coinvestigator on two National Institutes of Health grants (1R01HD086792-01 and U01HD076733-01A1) and has provided legal consulting for the law office of Lubin & Meyer, P.D. (Boston, Massachusetts).
References
Brennan, TA, Leape, LL . Adverse events, negligence in hospitalized patients: Results from the Harvard Medical Practice Study. Perspect Healthc Risk Manage. 1991; 11:2–8 [Article] [PubMed]
Weinger, MB, Slagle, J, Jain, S, Ordonez, N . Retrospective data collection and analytical techniques for patient safety studies. J Biomed Inform. 2003; 36:106–19 [Article] [PubMed]
Weinger, MB, Slagle, J . Human factors research in anesthesia patient safety. Proc AMIA Symp. 2001 756–60
Barach, P, Small, SD . Reporting and preventing medical mishaps: Lessons from non-medical near miss reporting systems. BMJ. 2000; 320:759–63 [Article] [PubMed]
Heard, GC, Sanderson, PM, Thomas, RD . Barriers to adverse event and error reporting in anesthesia. Anesth Analg. 2012; 114:604–14 [Article] [PubMed]
Forrest, JB, Cahalan, MK, Rehder, K, Goldsmith, CH, Levy, WJ, Strunin, L, Bota, W, Boucek, CD, Cucchiara, RF, Dhamee, S, Domino, KB, Dudman, AJ, Hamilton, WK, Kampine, J, Kotrly, KJ, Maltby, JR, Mazloomdoost, M, MacKenzie, RA, Melnick, BM, Motoyama, E, Muir, JJ, Munshi, C . Multicenter study of general anesthesia. II. Results. Anesthesiology. 1990; 72:262–8 [Article] [PubMed]
Oken, A, Rasmussen, MD, Slagle, JM, Jain, S, Kuykendall, T, Ordonez, N, Weinger, MB . A facilitated survey instrument captures significantly more anesthesia events than does traditional voluntary event reporting. Anesthesiology. 2007; 107:909–22 [Article] [PubMed]
Kingston, MJ, Evans, SM, Smith, BJ, Berry, JG . Attitudes of doctors and nurses towards incident reporting: A qualitative analysis. Med J Aust. 2004; 181:36–9 [Article] [PubMed]
Noble, DJ, Pronovost, PJ . Underreporting of patient safety incidents reduces health care’s ability to quantify and accurately measure harm reduction. J Patient Saf. 2010; 6:247–50 [Article] [PubMed]
Weinger, MB, Reddy, SB, Slagle, JM . Multiple measures of anesthesia workload during teaching and nonteaching cases. Anesth Analg. 2004; 98:1419–25, table of contents [Article] [PubMed]
Slagle, JM, Porterfield, ES, Lorinc, AN, Afshartous, D, Shotwell, MS, Weinger, MB . Prevalence of potentially distracting noncare activities and their effects on vigilance, workload, and nonroutine events during anesthesia care. Anesthesiology. 2018; 128:44–54 [Article] [PubMed]
Slagle, JM, Weinger, MB . Effects of intraoperative reading on vigilance and workload during anesthesia care in an academic medical center. Anesthesiology. 2009; 110:275–83 [PubMed]
Xu, J, Reale, C, Slagle, JM, Anders, S, Shotwell, MS, Dresselhaus, T, Weinger, MB . Facilitated nurse medication-related event reporting to improve medication management quality and safety in intensive care units. Nurs Res. 2017; 66:337–49 [Article] [PubMed]
Weinger, MB, Herndon, OW, Gaba, DM. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology. 1997; 87:144–55. discussion 29A–30A [Article] [PubMed]
Borg, G . Simple Rating Methods of Perceived Exertion, Physical Work and Effort. 1977 Permagon Press
Weinger, MB, Herndon, OW, Zornow, MH, Paulus, MP, Gaba, DM, Dallen, LT . An objective methodology for task analysis and workload assessment in anesthesia providers. Anesthesiology. 1994; 80:77–92 [Article] [PubMed]
Cao, CG, Weinger, MB, Slagle, J, Zhou, C, Ou, J, Gillin, S, Sheh, B, Mazzei, W . Differences in day and night shift clinical performance in anesthesiology. Hum Factors. 2008; 50:276–90 [Article] [PubMed]
Rayo, M, Smith, P, Weinger, MB, Slagle, J, Dresselhaus, T . Assessing medication safety technology in the intensive care unit. Proc Hum Factors Ergon Soc. 2007 692–6
Boëlle, PY, Garnerin, P, Sicard, JF, Clergue, F, Bonnet, F . Voluntary reporting system in anaesthesia: Is there a link between undesirable and critical events? Qual Health Care. 2000; 9:203–9 [Article] [PubMed]
Bothner, U, Georgieff, M, Schwilk, B . The impact of minor perioperative anesthesia-related incidents, events, and complications on postanesthesia care unit utilization. Anesth Analg. 1999; 89:506–13 [PubMed]
Cooper, JB, Newbower, RS, Kitz, RJ . An analysis of major errors and equipment failures in anesthesia management: Considerations for prevention and detection. Anesthesiology. 1984; 60:34–42 [Article] [PubMed]
Donchin, Y, Gopher, D, Olin, M, Badihi, Y, Biesky, M, Sprung, CL, Pizov, R, Cotev, S . A look into the nature and causes of human errors in the intensive care unit. Crit Care Med. 1995; 23:294–300 [Article] [PubMed]
Runciman, WB, Sellen, A, Webb, RK, Williamson, JA, Currie, M, Morgan, C, Russell, WJ . The Australian Incident Monitoring study. Errors, incidents and accidents in anaesthetic practice. Anaesth Intensive Care. 1993; 21:506–19 [Article] [PubMed]
Runciman, WB, Webb, RK, Lee, R, Holland, R . The Australian Incident Monitoring study. System failure: An analysis of 2000 incident reports. Anaesth Intensive Care. 1993; 21:684–95 [Article] [PubMed]
Yanes, AF, McElroy, LM, Abecassis, ZA, Holl, J, Woods, D, Ladner, DP . Observation for assessment of clinician performance: A narrative review. BMJ Qual Saf. 2016; 25:46–55 [Article] [PubMed]
Teutsch, SM, Churchill, RE . Principles and Practice of Public Health Surveillance. 2000. 2nd edition, New York: Oxford University Press
Slight, SP, Howard, R, Ghaleb, M, Barber, N, Franklin, BD, Avery, AJ . The causes of prescribing errors in English general practices: A qualitative study. Br J Gen Pract. 2013; 63:e713–20 [Article] [PubMed]
Kambakamba, P, Dindo, D, Nocito, A, Clavien, PA, Seifert, B, Schäfer, M, Hahnloser, D . Intraoperative adverse events during laparoscopic colorectal resection–better laparoscopic treatment but unchanged incidence. Lessons learnt from a Swiss multi-institutional analysis of 3,928 patients. Langenbecks Arch Surg. 2014; 399:297–305 [Article] [PubMed]
Phadnis, J, Templeton-Ward, O . Inadequate preoperative team briefings lead to more intraoperative adverse events. J Patient Saf. 2018; 14:82–6 [Article] [PubMed]
Cullen, DJ, Bates, DW, Small, SD, Cooper, JB, Nemeskal, AR, Leape, LL . The incident reporting system does not detect adverse drug events: A problem for quality improvement. Jt Comm J Qual Improv. 1995; 21:541–8 [PubMed]
Cook, RI, Render, M, Woods, DD . Gaps in the continuity of care and progress on patient safety. BMJ. 2000; 320:791–4 [Article] [PubMed]
Carayon, P, Hancock, P, Leveson, N, Noy, I, Sznelwar, L, van Hootegem, G . Advancing a sociotechnical systems approach to workplace safety–developing the conceptual framework. Ergonomics. 2015; 58:548–64 [Article] [PubMed]
Chandonnet, CJ, Kahlon, PS, Rachh, P, Degrazia, M, Dewitt, EC, Flaherty, KA, Spigel, N, Packard, S, Casey, D, Rachwal, C, Agrawal, PB . Health care failure mode and effect analysis to reduce NICU line-associated bloodstream infections. Pediatrics. 2013; 131:e1961–9 [Article] [PubMed]
Mackenzie, CF, Jefferies, NJ, Hunter, WA, Bernhard, WN, Xiao, Y . Comparison of self-reporting of deficiencies in airway management with video analyses of actual performance. LOTAS Group. Level One Trauma Anesthesia Simulation. Hum Factors. 1996; 38:623–35 [Article] [PubMed]
Neily, J, Silla, ES, Sum-Ping, SJT, Reedy, R, Paull, DE, Mazzia, L, Mills, PD, Hemphill, RR . Anesthesia adverse events voluntarily reported in the Veterans Health Administration and lessons learned. Anesth Analg. 2018; 126:471–7 [Article] [PubMed]
Percarpio, KB, Watts, BV, Weeks, WB . The effectiveness of root cause analysis: What does the literature tell us? Jt Comm J Qual Patient Saf. 2008; 34:391–8 [PubMed]
Buist, M, Bernard, S, Nguyen, TV, Moore, G, Anderson, J . Association between clinically abnormal observations and subsequent in-hospital mortality: A prospective study. Resuscitation. 2004; 62:137–41 [Article] [PubMed]
Pelter, MM, Adams, MG, Drew, BJ . Transient myocardial ischemia is an independent predictor of adverse in-hospital outcomes in patients with acute coronary syndromes treated in the telemetry unit. Heart Lung. 2003; 32:71–8 [Article] [PubMed]
Elder, NC, Vonder Meulen, M, Cassedy, A . The identification of medical errors by family physicians during outpatient visits. Ann Fam Med. 2004; 2:125–9 [Article] [PubMed]
Slagle, JM, Anders, S, Porterfield, E, Arnold, A, Calderwood, C, Weinger, MB . Significant physiological disturbances associated with nonroutine event containing and routine anesthesia cases. J Patient Saf. 2015; 11:198–203 [Article] [PubMed]
Weinger, MB, Gonzales, DC, Slagle, J, Syeed, M . Video capture of clinical care to enhance patient safety. Qual Saf Health Care. 2004; 13:136–44 [Article] [PubMed]
Fig. 1.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
Fig. 1.
Total case enrollment. The figure shows the relationship between the total case enrollment and the final case cohort used in the video review and analysis. *29 nonroutine event–containing cases were excluded from the final dataset because the events were not reviewable on video.
×
Fig. 2.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
Fig. 2.
A selected nonroutine event and its associated detailed description, contributory factors, and patient impact events.
×
Table 1.
Demographic Variables
Demographic Variables×
Demographic Variables
Table 1.
Demographic Variables
Demographic Variables×
×
Table 2.
Study Case and Surgical Variables
Study Case and Surgical Variables×
Study Case and Surgical Variables
Table 2.
Study Case and Surgical Variables
Study Case and Surgical Variables×
×
Table 3.
Contributory Factors in Nonroutine Event–Containing Cases
Contributory Factors in Nonroutine Event–Containing Cases×
Contributory Factors in Nonroutine Event–Containing Cases
Table 3.
Contributory Factors in Nonroutine Event–Containing Cases
Contributory Factors in Nonroutine Event–Containing Cases×
×