Free
Education  |   October 2012
Unanticipated Difficult Airway in Obstetric Patients: Development of a New Algorithm for Formative Assessment in High-fidelity Simulation
Author Affiliations & Notes
  • Mrinalini Balki, M.B.B.S., M.D.
    *
  • Mary Ellen Cooke, M.D., F.R.C.P.C.
  • Susan Dunington, M.A., R.R.T., F.C.S.R.T.
  • Aliya Salman, M.D., F.R.C.P.C.
    §
  • Eric Goldszmidt, M.D., F.R.C.P.C.
  • *Associate Professor, Department of Anesthesia and Pain Medicine, University of Toronto, Toronto, Ontario, Canada. Assistant Professor, Department of Anesthesia and Pain Medicine, University of Toronto. Faculty, Primary and Critical Care, The Michener Institute, Toronto, Ontario, Canada. §Lecturer, Department of Anesthesia and Pain Medicine, University of Toronto.
Article Information
Education / Airway Management / Obstetric Anesthesia
Education   |   October 2012
Unanticipated Difficult Airway in Obstetric Patients: Development of a New Algorithm for Formative Assessment in High-fidelity Simulation
Anesthesiology 10 2012, Vol.117, 883-897. doi:10.1097/ALN.0b013e31826903bd
Anesthesiology 10 2012, Vol.117, 883-897. doi:10.1097/ALN.0b013e31826903bd
What We Already Know about This Topic
  • Unanticipated difficult airway can occur in obstetric emergencies, yet little work has been done regarding education, especially using high-fidelity simulation, in handling this problem

What This Article Tells Us That Is New
  • A consensus-based algorithm and checklist analysis of high-fidelity simulation teaching of unanticipated difficult airway in obstetrics were developed and the latter was validated

  • Common critical errors were observed in simulation testing of residents, suggesting the need for focused education and possibly experiential learning through successive simulations

AN unanticipated difficult airway can be challenging in obstetric patients. There are a myriad of specific clinical variations of this problem, and no universally accepted algorithm or management consensus. “Anesthesia causes” remain on the list of top 10 direct causes of maternal mortality and airway-related deaths lead that category.1,2 The incidence of difficult intubation in obstetric patients is 1.3–16.3%,3  5 and that of failed intubation is 1:250 to 1:300, at least 10 times more than in the general surgical population.6,7 Maternal deaths from difficult airway management were also highlighted in two Reports of the Confidential Enquiries into Maternal Deaths in the United Kingdom (2006–2008 and 2000–2002).1,8 The American Society of Anesthesiologists closed claims in obstetrics database revealed that the most common claim, maternal death (22%), was more frequently associated with general than regional anesthesia, and 16% of claims associated with obstetric anesthesia care were due to critical events involving the airway and respiratory system.2 The potentially high maternal and fetal morbidity and mortality associated with poor management of the unanticipated difficult airway in obstetric patients warrants more attention to this possible catastrophe. The current infrequency of administration of general anesthesia in normal obstetric practice limits hands-on experience, and threatens the ability to maintain skill levels and clinical competency.
In the setting of obstetric emergencies requiring general anesthesia, the simulation model, referred to by the enthusiasts as an “ethical imperative,”9 may allow for improvement in critical skills10 and crisis resource management11 without compromising patient safety. A recent editorial states that “simulation aims to create a virtuous cycle of professional development to improve patient outcomes.”12 As noted by Boulet and Murray, “the construction of quality-based simulation assessment continues to be a difficult task. Important within this process is choosing the particular scenarios that offer the best opportunity to sample the knowledge and skills that one wishes to measure.”13 
Although simulation has been used in anesthesia in a wide range of emergencies, in many subspecialty disciplines, published reports on simulation for obstetric emergencies are rare14  17 and focus mainly on issues related to team communication.16  19 Our literature search revealed a lack of research using simulator scenarios for specifically assessing the critical skills in overall management of unanticipated difficult intubation in obstetrics. Moreover, there are no widely accepted algorithms or standardized checklists on the unanticipated difficult airway in obstetrics to provide guidance and for objective assessment of the critical skills required to manage such cases.20  22 
The primary objective of this study was to develop a consensus-based algorithm for the management of the unanticipated difficult airway in obstetric patients. This new algorithm was then used as the basis for the assessment of senior anesthesia residents' performance using high-fidelity simulation in four specific scenarios. This research represents a formative assessment of residents' management and decision-making. Formative assessments are used to aid learning and have been described as “assessment for  learning.”23 To be useful, feedback from formative assessment needs to occur in a timely manner, which we endeavored to do in this study.
Materials and Methods
Design and Recruitment of Participants
This was a prospective, observational cohort study. After Institutional Review Board approval at Mount Sinai Hospital, all senior anesthesia residents in postgraduate years (PGY) 4 and 5 from the University of Toronto were invited to participate in the study. Written informed consent was obtained from the participants for videotaping and assessment of their performance. A confidentiality agreement was signed confirming that they would not discuss the details of clinical scenarios with anyone before the end of the study. The participants were unaware of the nature of simulation and the scenario contents. The simulation sessions proceeded in the following manner: orientation, simulation, debriefing, and survey completion. Before the simulation sessions, each resident underwent an orientation to the mock labor and delivery operating room, mannequin, and monitors, as well as the method of debriefing and evaluation.
Simulation Center
The simulation sessions were held at The Michener Institute for Applied Health Sciences, in Toronto, Ontario, Canada. The simulation center was set up as a virtual obstetric operating room equipped with a Laerdal SimMan® simulator mannequin, anesthesia machine (Datex Corporation, St. Laurent, Quebec, Canada), monitors (Laerdal Medical Canada Ltd., Toronto, Canada) and airway equipment, along with the instruments needed to perform a cesarean section (CS). The airway equipment consisted of facemasks, oropharyngeal airways, laryngeal mask airways (LMA) (classic, proseal, and intubating), endotracheal tubes of various sizes, short-handled laryngoscopes with Macintosh blades, bougie, and glidescope, whereas the difficult airway cart contained alternative laryngoscope blades—Miller and McCoy, cricothyrotomy kit (Cook Medical Inc., Bloomington, IN) and fiberoptic bronchoscope (Olympus, Richmond Hill, Ontario, Canada). To optimize the realism of the pregnant model, an abdominal flap was fit over the SimMan abdomen, which could be lifted for the delivery of the neonate and the placenta. The airway of the mannequin could be modified to suit the scenario description. Digital videography was used to record the scenarios with superimposed vital signs on the image, and these recordings were used for debriefing purposes and evaluation by the raters.
Development of Algorithm, Checklist, and Scenarios
An algorithm for unanticipated difficult airway management in obstetrics was created by the study investigators based on previously published algorithms, literature review,2022,24,25 and their own experience and teaching. Four of these investigators subspecialize in obstetric anesthesia and teach extensively at junior resident, senior resident, and fellowship levels. Two of them are past and current oral examination board members in anesthesiology for the Royal College of Physicians and Surgeons of Canada. One investigator has two publications on the obstetric airway.26,27 The modified Delphi technique was used to obtain a broader consensus on the algorithm items and layout.28 We discussed our initial version of the algorithm in a focus group of other obstetric anesthesiologists within our hospital department. It was also presented at University of Toronto research rounds and another forum by the principal investigator. It was then sent out electronically for external review to seven experts in obstetric anesthesia, of whom five participated in the process. These experts are members of the Society for Obstetric Anesthesia and Perinatology and/or the Canadian Anesthesiologists' Society, and represent several geographic areas of Canada. The experts were blinded to each other's identities. They were asked to make pertinent comments and modify the algorithm as deemed appropriate by suggesting the elimination or addition of steps, and to provide explanation or evidence for any suggested changes. Their suggestions were incorporated and the revised algorithm was circulated for further comments. The final algorithm generated through this process after consensus from all the members after two rounds of Delphi formed our gold standard for management (fig. 1). A performance checklist was then developed for each step of this algorithm by the investigators.
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
×
The algorithm itself covers management of six generic situations and delineates three different management pathways. The six generic situations were derived from the three contexts of the unanticipated difficult airway in obstetrics: elective CS, emergency CS for fetal distress, and emergency CS for maternal distress, and the two possibilities within each context: can ventilate and cannot ventilate. Of these six situations, we chose four specific realistic clinical variations for the four different simulator scenarios (appendix 1). Each scenario was intended to be managed down one of the three different management pathways so that all three critical care management strategies in the algorithm could be assessed. The three management pathways are: proceed to CS under general anesthesia without an endotracheal tube; awaken mother and do either awake intubation or regional; or do transtracheal airway and proceed immediately to CS.
Just before each simulation session, the residents received a one-page synopsis of the usual preoperative assessment of the mother, the reasons for CS, and the rationale for general anesthesia as well as the initial setup (left uterine displacement, intravenous access in situ  , basic monitors applied, etc  .). The four scenarios were:
Scenario 1.
Fetal emergency (cord prolapse)—critical event involving unanticipated difficult intubation and cannot ventilate.
Scenario 2.
Maternal emergency (massive antepartum hemorrhage and fetal distress)—critical event involving unanticipated difficult intubation, can ventilate, and ongoing maternal hemodynamic instability.
Scenario 3.
Fetal emergency (ruptured vasa previa)—critical event involving unanticipated difficult intubation and can ventilate.
Scenario 4.
Elective CS under general anesthesia (regional technically impossible due to previous spinal surgery)—critical event involving unanticipated difficult intubation and can ventilate.
We decided to order these four scenarios as above and not attempt to order them from our perspective of difficulty. However, we had some preconceived notions as to which scenario might be most stressful (scenario 1), most complex and challenging to handle (scenario 2), present the ethical conundrum of risking maternal aspiration for the sake of a compromised fetus (scenario 3), and theoretically easiest, though most likely to trip up a resident should he or she forget the context is elective (scenario 4).
These four specific scenarios were designed by the investigators and thought to be clinically relevant by the focus group of obstetric anesthesiologists at our institution. The scenarios were based on the real-life possibility of a combination of an unanticipated difficult airway with a common or classic obstetric situation. This rare but potentially catastrophic combination is invariably covered at length in standard subspecialty textbooks. We also cover these scenarios and variations of them in our own teaching and practice oral sessions. Their feasibility was tested in the simulation setting before participants' assessment to ensure that they would reflect the skills and actions required by anesthesiologists in difficult airway management in obstetrics.
The standard settings on the mannequin included the application of tongue edema to ensure a Cormack and Lehane grade 4 view on laryngoscopy. If oxygen delivery was interrupted for more than 20 s or longer, oxygen saturation dropped by 2% every 5 s after the first failed intubation attempt, and every 3 s after the second and subsequent intubation attempts. Oxygen saturation was restored at a rate of 5% every 2 s after restoration of ventilation. The decrease or increase in oxygen saturation was associated with corresponding change in the pitch of the oximeter to create a sense of realism.
An investigator (S.D.), with extensive experience in simulator programming, conducted the sessions following the specific flowsheets that were generated by the investigators with respect to various possible management options within each of the four scenarios (appendix 1). The program's timer was used to count the seconds between each interval to ensure consistency. A pilot video recording of all four sessions was performed to confirm all the listed items in the checklist could be identified during video review.
All residents, during their orientation, were specifically instructed that they should perform all actions as in a real-life case, as well as verbalize their thoughts and plans to facilitate appropriate evaluation. Each resident performed as a primary anesthesiologist and underwent four simulation sessions in sequence, with a common team of trained actors playing the scripted role of an obstetrician, nurse, and respiratory therapist. The programmer and actors were provided with guidelines including how often and what verbal responses should be provided real-time either voluntarily or in response to a participant's question. Each simulation scenario lasted approximately 8–10 min. After each scenario, the resident completed a short questionnaire to rate the orientation, their opinion of the importance of the scenario, and their own performance.
No debriefing or feedback was provided between simulation scenarios. One debriefing session was conducted by an investigator at the end of all four simulations in a standardized manner to provide performance feedback to the resident. As part of the debriefing, a copy of the algorithm was used to review appropriate management of each scenario. At the end of the debriefing, the residents were requested to fill out an additional questionnaire on their previous reading and teaching on this subject, and to provide more feedback on our simulations and on the debriefing session itself (appendix 2). Last, the residents were provided with a substantial teaching handout containing a one-page narrative overview of our new algorithm, relevant North American and British literature giving historical context, examples of previous algorithms, and addressing some controversies in the management of unanticipated difficult airway in obstetrics. In addition, they were directed to read the relevant chapters from two subspecialty textbooks. The entire study session was completed within 2 h.
Evaluation Metrics
Three raters, blinded to the residents' levels of training, independently evaluated the critical and crisis resource management skills on the videotaped sessions by use of our designed checklist and previously validated Ottawa Global rating scale (GRS), respectively. The checklist consisted of 15 items rated on an anchored ordinal scale (0–2; 0 = action not performed, 1 = performed but not timely, 2 = performed timely). The GRS was rated on 7-point Likert scale (1–7; where 1 = poor and 7 = superior).29,30 The raters had expertise in obstetric anesthesia and previous experience in both simulation development and assessment of videotaped sessions. As part of this project, they met to discuss the objectives and specific applications of the defined scoring rubric, both for the checklist and the GRS, based on the scenarios. They observed three pilot scenarios covered in the study, decided on the expectations, and discussed their ratings to produce an agreed reference rating. No formal attempt was made to calibrate the raters. Thereafter they independently reviewed and rated all the study videos. They followed the sequence of events in the algorithm/checklist while rating for timeliness. The practice videos did not have any study participants.
Our primary outcome was the evaluation of residents' performance based on the critical skills checklist derived from the new algorithm, whereas the secondary outcomes consisted of GRS scores, the difference in scores based on the level of residency, identification of common critical errors, and participants' feedback.
Statistical Analysis
We measured the average or median values of scores determined by the three reviewers. The frequencies and proportions of tasks performed, and the geometric mean and SD of the scores in percentage points were calculated. Nonparametric Spearman correlation was used to examine the association between total checklist and GRS scores. Generalized estimating equations were used to examine the relationship between technical skills and PGY status while accounting for the intraparticipant correlation. To measure the consistency across raters, intraclass correlation coefficients [ICC (3, 1)]31 and 95% CIs were calculated for each scenario and item within the checklist and GRS to evaluate interrater reliability. In addition to the classic reliability analysis, generalizability theory was used to determine the reliability of the participant scores and identify variance components, such as rater or scenario that best explained the variability in individual performance. Finally, scenario discrimination statistics (correlation between a participant's mean scenario score and his or her overall score on the allotted scenarios) were used to investigate the validity of the scoring on scenarios (i.e.  , whether participants who scored highly on one scenario scored highly on all scenarios and vice versa  ). P  < 0.05 was considered statistically significant. All analyses were completed using SAS Version 9.2 (Cary, NC).
Using the method outlined by Walter et al.  ,32 assuming a type I error of 0.05, a type II error of 0.20, and three raters per subject, we determined that 11 participants would be required to identify moderate ICC coefficients (0.50) or higher as statistically greater than 0.0. To ensure the robustness of the secondary aims (i.e.  , evaluating differences between PGY 4 and PGY 5 status) we aimed to recruit 20 participants.
Results
The study recruitment was done between March and November 2010. Forty senior anesthesia residents were invited and 16 agreed to participate in the study. There were seven PGY4 and nine PGY5 residents. All residents had completed their obstetric anesthesia rotation and had participated in routine nonobstetric simulation sessions that are held once a year in the first and second year of residency. Each resident completed 4 simulation sessions in the same sequence, and video recordings of all 64 simulation scenarios were analyzed.
Common Performance Errors
The proportion and frequency of tasks performed by the participants in all scenarios are shown in table 1. Several critical errors were identified. The common tasks that were not done at all or not done in a timely manner by approximately 50% of participants included (items 4, 5, 7, 8, 10, and 14): calling for skilled help or difficult airway cart; using an alternative approach for intubation during subsequent attempts or considering LMA placement; recognizing inadequate oxygenation (scenario 1); and considering definitive airway postdelivery if the mother was unstable (scenario 2) (table 1). LMA placement was considered as one of the airway options in only 47% of scenarios (30 of 64), and more than three attempts at intubation were tried in 13% (8 of 64) scenarios (data not shown).
Table 1. Percentage of Participants Completing the Critical Skills Checklist in Each Scenario
Image not available
Table 1. Percentage of Participants Completing the Critical Skills Checklist in Each Scenario
×
Critical Skills Assessment
The participants' critical skills checklist scores in each scenario are shown in table 2. The average scores across scenarios ranged from 64–80% (fig. 2). The cumulative mean (SD) score of all residents was 73% (1.22). The mean change in scores over time was 15.4% (19.8%). There was no significant difference in the GRS scores between different scenarios, although the scores were the lowest in scenario 2 and highest in scenario 4 (table 3). The examinee's scores were consistent across the scenarios both for the checklist [ICC (3,1) = 0.86; 95% CI: 0.68, 0.95] and the GRS [ICC (3,1) = 0.95; 95% CI: 0.80, 0.99]. There was no significant difference in the overall checklist scores [72.10% (1.32) vs.  73.85% (1.22), P  = 0.69] (table 2) or the mean GRS scores [5.15 (0.73) vs.  5.32 (0.64), P  = 0.57] between PGY4 and PGY5 residents (fig. 2). However, PGY5 residents demonstrated significantly superior performance in most of the critical tasks (items 2, 4, 6, 9–13) compared with PGY4 residents.
Table 2. Critical Skills Checklist Scores
Image not available
Table 2. Critical Skills Checklist Scores
×
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
×
Table 3. Global Rating Scale Scores
Image not available
Table 3. Global Rating Scale Scores
×
Reliability and Validity
The scoring checklist demonstrated a high interrater reliability among the three raters with an overall ICC of 0.69 (95% CI: 0.58, 0.78). The ICC for various components of the GRS items was moderate at 0.49 (95% CI: 0.33, 0.64). The combined ICC for both checklist and GRS rating was 0.64 (95% CI: 0.52, 0.74). However, the use of classic test theory (e.g.  , ICCs) has some disadvantages when examining reliability, in that it is unable to identify which sources are contributing to variation in scores. Generalizability theory addresses some of these concerns33,34 and was thus performed. Partitioning out of the variance components demonstrated that the raters contributed relatively little to the variance in participant scores, whereas the participants and scenarios (and correspondingly the interaction) explained the most variation in scores (table 4). With moderate to high generalizability (G) coefficients of 0.58 and 0.70, the checklist and GRS scales demonstrated reasonable reliability. Furthermore, discrimination analysis was conducted to assess the potential validity of the scenarios, correlating individual scenario scores to the total score across all scenarios. The discrimination coefficient was moderately high for both scales, suggesting that participants who scored well on one scenario scored well on the others and vice versa  (table 4).
Table 4. Generalizability and Discrimination Analysis of Checklist and Global Rating Scale Scores Across Four Scenarios Rated by Three Raters
Image not available
Table 4. Generalizability and Discrimination Analysis of Checklist and Global Rating Scale Scores Across Four Scenarios Rated by Three Raters
×
Correlations between Categories of Performance
There was moderate correlation between checklist and GRS scores of the residents (r = 0.49; P  < 0.001). There was no significant correlation between participants' rating of how well this topic was covered in their previous formal or informal teaching with their overall scores (r = 0.32; P  = 0.23). Only 38% of students had read the obstetric anesthesia textbook chapter on difficult airway management in obstetrics. All but one resident stated that they had never experienced unexpected difficult airway in obstetrics. Therefore, we could not evaluate whether or not previous experience was correlated with their score.
Participants' Feedback
On average, the participants rated the setup and realism of scenarios as 3.4 (0.6), their importance in clinical practice as 3.8 (0.4), and debriefing as 3.8 (0.5) on a 4-point scale (1–4; 1 = poor and 4 = excellent). The visual and verbal setup was deemed adequate by 81% of participants, and all of them mentioned that all components of the scenarios were easy to understand. All the participants found scenario 2 the most stressful and challenging, and rated themselves poorly on it [2.0 (0.5)] compared with other scenarios [scenario 1 = 2.7 (0.6), scenario 3 = 2.8 (0.5), scenario 4 = 3.1 (0.57)]. All stated that our algorithm, which was used during debriefing, would have helped them improve performance in these cases and clearly expressed its potential use in clinical case management.
Discussion
Our new algorithm was developed to help us provide a gold standard by which to assess the clinical judgment and critical skills of anesthesia residents in the management of unanticipated difficult airway in obstetrics in the simulator setting. Our results suggest that high-fidelity simulation may prove to be an effective tool for formative assessment of the management of this potential catastrophe. The algorithm and associated simulations may also translate into better real-life practice.
In 1976, Tunstall et al.  made pioneering attempts at developing a specific obstetric algorithm.35 Since that time, there have been many other proposed algorithms with various modifications and evolutions.2022,24,25,36  40 Any new algorithm must address both the unique situation of two patients and an almost infinite number of possible specific clinical variations, which relate to elective situations and two types of emergency situations, i.e.  , fetal and maternal. One must also be cognizant of the fact that any maternal emergency has a very high likelihood of soon becoming a fetal emergency too (i.e.,  a combined maternal and fetal emergency). Finally, one must strive to offer practical, specific, clear-cut management advice with a limited number of strategies, so they can be remembered and efficiently executed in this stressful and unexpected circumstance.
We created a new algorithm (fig. 1) after careful study of these previous algorithms, review of our own clinical experience, and with the input of other experts. We thought that the best way to approach the multiple issues in this type of algorithm was to define three contexts—elective CS, CS for fetal emergency, and CS for maternal emergency, and to recognize the two possibilities of can and cannot ventilate within these contexts. This represents only six generic situations under which most of the clinical variations of CS under general anesthesia can be categorized. An important component of this new algorithm is that after the common initial steps, there are essentially just three management pathways. This is because three of the six generic situations are cannot ventilate situations and go down the same pathway, i.e.  , they lead to a transtracheal airway and to immediate CS.
The three other generic situations are “can ventilate” ones and have two alternative pathways between them: proceed without an endotracheal tube or awaken the patient (then choose awake intubation or regional). Elective CS should be handled by awakening the mother. The situation of fetal emergency may present an ethical conundrum regarding risking the mother's life to allow for the delivery of a distressed fetus (fig. 1, superscript 5). The situation of “maternal distress and can ventilate” is complex, as the mother is unstable and there will be impending fetal distress. Therefore, CS should proceed for the sake of both patients. If maternal instability persists after fetal delivery, then a definitive airway should be reconsidered using a different approach.
Our algorithm does not directly address the context of semiurgent CS for maternal or fetal issues, but does give management guidance for these cases. The last directive in the third box of the initial common steps, clarify urgency, is vital in terms of critical decision-making for these cases. It is meant to spark a necessary, rapid, and focused interdisciplinary exchange between anesthesiologist and obstetrician. The communication should lead to reclassifying the case as essentially elective or an impending true emergency (fig. 1, superscript 2). In the rare instance where a case cannot be readily or agreeably reclassified, the anesthesiologist will at least be properly informed to make their own decision between proceeding or awakening.
With respect to issues of management consensus, we hope our algorithm will generate discussion. However, there are points within it where consensus may neither be possible nor advisable. For instance, our algorithm is designed to allow for two options in the situation of fetal emergency and can ventilate. As well our seven superscripts in the algorithm (fig. 1) speak to areas that may be open to case specific personal discretion.
Our algorithm provided good face and content validity as determined by internal and external experts and buy-in by our residents at verbal debriefing and via  final questionnaire. The psychometric analysis of the participants' scores suggests that these scenarios can be reliably used and are valid measures to assess obstetric airway management.
We used the checklist as an explicit process for objective assessment of critical skills and GRS as an implicit process for assessment of qualitative skills of crisis resource management.13,23 A moderate correlation between these two scales indicates that acquisition of both skills can be complementary for improving performance under such circumstances.
All items in our checklist were equally weighted due to equal importance given by the experts for each step of the algorithm. We considered timeliness of the checklist items while scoring, as we thought that logical, sequential, and timely critical decisions are crucial in determining the outcome. Our checklist and GRS scores were consistent across scenarios and demonstrated good interrater reliability, similar to other studies.29,30 
The results of our study showed an improvement in scores of the critical skills checklist from the first to last scenario without debriefing after each scenario. There are at least two possible reasons for this improvement. One is reinforcement and experiential learning due to repeated exposure, and the second is that the difficult scenarios were performed first (as predicted by us and as performed by the residents). In the future, we may conduct simulations in the order of predicted difficulty (from easy to difficult) or with the same level of difficulty to lessen the confounders and to confirm if there is experiential learning.
The GRS scores were lowest in scenario 2, indicating the need for more training in dealing with this complex combined maternal and fetal emergency. The residents also rated themselves most poorly in this scenario, suggesting they found it very challenging to manage. The poor checklist scores in scenario 1 indicate the need for early recognition of a “cannot intubate, cannot ventilate” situation.
It was concerning to see that approximately half the number of participants did not consider calling for skilled help or a difficult airway cart. Moreover, in 13% of the scenarios, the participants tried multiple intubation attempts, failing to stop even after three attempts. In the current Report of the Confidential Enquiries into Maternal Deaths in the United Kingdom 2006–2008, an airway-related death involved persistence at intubation even though oxygenation was achieved through an intubating LMA.1 There is no consensus regarding the number of intubation attempts that can safely be tried before stopping and choosing an alternative airway device.20,21,24,25 It seems prudent to allow two attempts at intubation in pregnant patients as per the Canadian Airway Focus Group22; however, if ventilation is possible, our algorithm reflects that a third attempt is reasonable if the airway has not been traumatized, adequate oxygenation is being maintained, and there is a high likelihood of success. It was interesting to note that LMA placement was considered as one of the airway options in only 47% of scenarios, despite the fact that the LMA is a recommended obstetric airway rescue device for failed intubation and difficult ventilation in subspecialty textbooks, previous algorithms, and our own teaching.24,41 
There are some limitations of our study. First, our study has a small sample size, which limits the generalizability of the findings. Furthermore, it is unknown how experts would perform when confronted with identical simulation scenarios. Nevertheless, in the absence of any previous studies, this preliminary study may serve as a foundation for larger studies.
Second, our checklist requires more reliability and validation to be considered as a tool for “summative assessment” to define competence or track progress.13 Validity of the checklist can be improved by subjecting it to the Delphi technique, as was done with the algorithm. The reliability may be improved further with more extensive standardized training of the raters. This study was conducted with senior residents only. To establish construct validity of our checklist, future examination could include more junior residents in whom lower scores and greater variability is expected.
Finally, we did not evaluate the retention of knowledge acquired during simulation and debriefing, which would have helped us assess if repeated sessions help reduce the performance errors. As with all simulations, the artificial environment may be the reason for some of the common failures or errors such as failing to call for help.
In summary, we have designed a new algorithm, created a checklist from it, and run four consecutive simulations on the unanticipated difficult airway in obstetrics. We identified several frequent critical errors in residents' management of the unanticipated difficult airway in obstetric patients and thus areas for improved teaching. The areas that need special attention include calling for skilled help, calling for the difficult airway cart, limiting the number of attempts at intubation, considering the use of an LMA, and reconsidering a definitive airway if the mother continues to be unstable after the delivery of the fetus. These management errors and teaching gaps should be addressed both by didactic teaching and simulation sessions. We hope our algorithm and checklist can be used for future didactic teaching, as well as for formative and potentially summative assessment of residents' performance in the simulation setting.
The authors acknowledge Magda Erik-Soussi, M.Sc., Research Assistant, Department of Obstetric Anesthesia, Mount Sinai Hospital, Toronto, Ontario, Canada, for coordinating the simulation sessions and organizing the database for this study, and J. Charles Victor, M.Sc., Epidemiologist, Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada, for providing statistical support in data analysis for this study. The authors wish to thank Viren Naik, M.D. Med., Associate Professor, Department of Anesthesia, Director, Ottawa Skills and Simulation Centre, The Ottawa Hospital, Ottawa, Ontario, Canada, for helping with the initial planning of this study. Finally, the authors thank all the physician actors who were committed for role play during simulations and all the experts who participated in the development of the algorithm.
References
Centre for Maternal and Child Enquiries (CMACE): Saving mothers' lives: Reviewing maternal deaths to make motherhood safer: 2006–08. The Eighth Report on Confidential Enquiries into Maternal Deaths in the United Kingdom. BJOG 2011; 118(Suppl 1):1–203
Ross BK: ASA closed claims in obstetrics: Lessons learned. Anesthesiol Clin North America 2003; 21:183–97
McDonnell NJ, Paech MJ, Clavisi OM, Scott KL, ANZCA Trials Group: Difficult and failed intubation in obstetric anaesthesia: An observational study of airway management and complications associated with general anaesthesia for caesearean section. Int J Obstet Anesth 2008; 17:292–7
McKeen DM, George RB, O'Connell CM, Allen VM, Yazer M, Wilson M, Phu TC: Difficult and failed intubation: Incident rates and maternal, obstetrical, and anesthetic predictors. Can J Anaesth 2011; 58:514–24
Tsen LC, Pitner R, Camann WR: General anesthesia for cesarean section at a tertiary care hospital 1990–1995: Indications and implications. Int J Obstet Anesth 1998; 7:147–52
Lyons G: Failed intubation. Six years' experience in a teaching maternity unit. Anaesthesia 1985; 40:759–62
Hawthorne L, Wilson R, Lyons G, Dresner M: Failed intubation revisited: 17-yr experience in a teaching maternity unit. Br J Anaesth 1996; 76:680–4
The Confidential Enquiries into Maternal and Child Health. Why Mothers Die 2000–2002. The Sixth Report on the Confidential Enquiries into Maternal Deaths in the United Kingdom. London: RCOG Press, 2004
Cumin D, Merry AF, Weller JM: Standards of simulation. Anaesthesia 2008; 63:1281–4
Lipman S, Carvalho B, Brock-Utne J: The demise of general anesthesia in obstetrics revisited: Prescription for a cure. Int J Obstet Anesth 2005; 14:2–4
Douglas MJ, Preston RL: The obstetric airway: Things are seldom as they seem. Can J Anaesth 2011; 58:494–8
Fox R, Walker J, Draycott T: Medical simulation for professional development—science and practice. BJOG 2011; 118(Suppl 3):1–4
Boulet JR, Murray DJ: Simulation-based assessment in anesthesiology:Requirements for practical implementation. ANESTHESIOLOGY 2010; 112:1041–52
Scavone BM, Sproviero MT, McCarthy RJ, Wong CA, Sullivan JT, Siddall VJ, Wade LD: Development of an objective scoring system for measurement of resident performance on the human patient simulator. ANESTHESIOLOGY 2006; 105:260–6
Goodwin MW, French GW: Simulation as a training and assessment tool in the management of failed intubation in obstetrics. Int J Obstet Anesth 2001; 10:273–7
Morgan PJ, Pittini R, Regehr G, Marrs C, Haley MF: Evaluating teamwork in a simulated obstetric environment. ANESTHESIOLOGY 2007; 106:907–15
Daniels K, Lipman S, Harney K, Arafeh J, Druzin M: Use of simulation based team training for obstetric crises in resident education. Simul Healthc 2008; 3:154–60
Cass GK, Crofts JF, Draycott TJ: The use of simulation to teach clinical skills in obstetrics. Semin Perinatol 2011; 35:68–73
Siassakos D, Bristowe K, Hambly H, Angouri J, Crofts JF, Winter C, Hunt LP, Draycott TJ: Team communication with patient actors: Findings from a multisite simulation study. Simul Healthc 2011; 6:143–9
Henderson JJ, Popat MT, Latto IP, Pearce AC, Difficult Airway Society: Difficult Airway Society guidelines for management of the unanticipated difficult intubation. Anaesthesia 2004; 59:675–94
American Society of Anesthesiologists Task Force on Management of the Difficult Airway: Practice guidelines for management of the difficult airway: An updated report by the American Society of Anesthesiologists Task Force on Management of the Difficult Airway. ANESTHESIOLOGY 2003; 98:1269–77
Crosby ET, Cooper RM, Douglas MJ, Doyle DJ, Hung OR, Labrecque P, Muir H, Murphy MF, Preston RP, Rose DK, Roy L: The unanticipated difficult airway with recommendations for management. Can J Anaesth 1998; 45:757–76
Bould MD, Crabtree NA, Naik VN: Assessment of procedural skills in anaesthesia. Br J Anaesth 2009; 103:472–83
Thomas JA, Hagberg CA: The difficult airway: Risks, prophylaxis, and management, Chestnut's Obstetric Anesthesia: Principles and Practice, 4th edition. Edited by Chestnut DH, Polley LS, Tsen LC, Wong CA. Philadelphia, Mosby Elsevier Inc., 2009, pp 651–75
Stackhouse RA, Bainton CR: Difficult airway management, Shnider & Levinson's Anesthesia for Obstetrics, 4th edition. Edited by Hughes SC, Levinson G, Rosen MA. Philadelphia, Lippincott Williams & Wilkins, 2002, pp 375–88
Goldszmidt E: Is there a difference between the obstetric and non-obstetric airway? In: Evidence-based Obstetric Anesthesia. Edited by Halpern S, Douglas JM. Malden, Massachusetts, Blackwell Publishing Ltd., 2005, pp 225–36
Goldszmidt E: Principles and practices of obstetric airway management. Anesthesiol Clin 2008; 26:109–25
Clayton MJ: Delphi: A technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 1997; 17:373–86
Kim J, Neilipovitz D, Cardinal P, Chiu M: A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”). Simul Healthc 2009; 4:6–16
Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J: A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med 2006; 34:2167–74
Shrout PE, Fleiss JL: Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 1979; 86:420–8
Walter SD, Eliasziw M, Donner A: Sample size and optimal designs for reliability studies. Stat Med 1998; 17:101–10
Cronbach LJ, Nageswari R, Gleser GC: Theory of generalizability: A liberation of reliability theory. Br J Statistical Psychology 1963; 16:137–63
Shavelson RJ, Webb NM: Generalizability Theory: A Primer. Thousand Oaks, CA,Sage, 1991
Tunstall ME: Failed intubation in the parturient. Can J Anaesth 1989; 36:611–3
Harmer M: Difficult and failed intubation in obstetrics. Int J Obstet Anesth 1997; 6:25–31
Morris S: Management of difficult and failed intubation in obstetrics. Br J Anaesth 2001; 4:117–21
Kuczkowski KM, Reisner LS, Benumof JL: Airway problems and new solutions for the obstetric patient. J Clin Anesth 2003; 15:552–63
Russell R: Failed intubation in obstetrics: A self-fulfilling prophecy? Int J Obstet Anesth 2007; 16:1–3
Vasdev GM, Harrison BA, Keegan MT, Burkle CM: Management of the difficult and failed airway in obstetric anesthesia. J Anesth 2008; 22:38–48
Mhyre JM, Healy D: The unanticipated difficult intubation in obstetrics. Anesth Analg 2011; 112:648–52
Appendix 1: Scenarios for Unanticipated Difficult Airway in Obstetrics Scenario 1
You are the staff anesthesiologist covering labor and delivery. You are called to the operating room for an emergency cesarean section. 
Your obstetric colleague arrives now and says that his patient is a healthy G3P2 in early labor at term. There has been sudden and now unrelenting severe fetal bradycardia. 
The obstetrician says: “Mother is stable. Diagnosis is prolapsed cord.  Cannot reduce and cannot elevate fetal head. There is no time for spinal.” 
Your assessment reveals: mother stable and alert, no medications, no allergies, no past medical history, no problems during prior general anesthetics and normal airway parameters.
Machine is checked, and drugs, endotracheal tube, suction are ready. All monitors are on. Respiratory therapist has wedged the patient and is now preoxygenating.
Obstetrician preps and drapes the patient. Maternal oxygen saturation is 99%, blood pressure is 125/80 mmHg and heart rate is 110/min. Fetal heart rate is 40/min. 
NOW:  You must induce the patient. Respiratory therapist is ready to apply cricoid pressure.
Refer to figure 3for programming this scenario.
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
×
Scenario 2
You are the staff anesthesiologist covering labor and delivery. You are called for a STAT cesarean section. 
Your obstetric colleague tells you that this is a patient with Placental Abruption. 
She is a pregnant woman who has not yet delivered (G1P0) at 39/40 weeks who just presented to triage with abdominal pain and vaginal bleeding. Estimated blood loss is up to 2 l  . Her vital signs were: blood pressure 90/60 mmHg, heart rate 120/min and oxygen saturation 97% on room air.
Your assessment reveals: she is a smoker but has no other significant past history, no allergies, no medications, no problems with prior general anesthetics. All her airway parameters are normal. She is feeling dizzy.
The nurse has placed a 16 G IV cannula and sent STAT blood for baseline complete blood count, coagulation work, and immediate cross-match of 4 units.
Respiratory therapist efficiently prepares the patient—wedge, monitors, suction, drugs, endotracheal tube are ready. Pentaspan is running  through blood warmer, arterial line is inserted  , preoxygenation is being done. Blood is on the way. Abdomen is prepped and draped.
There is ongoing vaginal bleeding. Mother's vitals now are: blood pressure 80/60 mmHg, heart rate 120/min, oxygen saturation 97%.
Fetal heart rate is 40/min. Obstetrician says it is an absolute emergency.
NOW:  You must induce the patient. Respiratory therapist is ready to apply cricoid pressure.
Refer to figure 4for programming this scenario.
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Scenario 3
You are the staff anesthesiologist covering labor and delivery.
You are called for a STAT caesarean section  for ruptured Vasa Previa  in a healthy 42-yr- old, G1 P0 at 38 weeks gestation. The obstetrician says this is an absolute fetal emergency  .
Your assessment reveals: stable vital signs, no past medical history, no medications, no allergies, past general anesthesia for tonsillectomy without any problems. All airway parameters are normal.
Fetal heart rate is 40/min.
Your operating room was fully set up this morning for any emergency cesarean section. Your respiratory therapist has wedged the patient, applied monitors and is preoxygenating.
Mother has a good running IV. Her vital signs remain normal. The abdomen is prepped and draped.
Fetal heart rate is now 20/min.
Now:  You must induce the patient. Respiratory therapist is ready to apply cricoid pressure.
Refer to figure 5for programming this scenario.
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Scenario 4
You are the staff anesthesiologist booked in the elective cesarean section  list.
The first patient is a G1 having a cesarean section  because of breech presentation.
You saw her in preoperative clinic. She is essentially healthy with no allergies, no medications and many past anesthetics in childhood without complications. Her airway parameters are all normal.
However, she does have extensive lumbar vertebral abnormalities.  She was born with major spina bifida and myelomeningocele. She has had multiple spinal operations including hardware insertion and bone grafting. She has extensive scarring from T10-S1 and no palpable interspaces. Your ultrasound examination could not reveal any normal anatomy or classic ultrasound patterns. Therefore, general anesthesia was agreed upon. Antacid prophylaxis was ordered and given.
Machine is checked, and drugs, endotracheal tube, and suction are ready. All monitors are on. Respiratory therapist has wedged her and is preoxygenating. Obstetrician has prepped and draped the abdomen.
She has normal vitals and good oxygen saturation. The fetal heart rate is normal.
Now:  You must induce the patient. Respiratory therapist is ready to apply cricoid pressure.
Refer to figure 6for programming this scenario.
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Appendix 2-a. SIMULATION Study Survey Participant #: __________ Date: __________
Image not available
Appendix 2-a. SIMULATION Study Survey Participant #: __________ Date: __________
×
Appendix 2-b. SIMULATION Study Survey Participant #: __________ Date: __________
Image not available
Appendix 2-b. SIMULATION Study Survey Participant #: __________ Date: __________
×
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
Fig. 1. Algorithm for unanticipated difficult airway in obstetric patients. BP = blood pressure; BMV = bag and mask ventilation; CS = cesarean section; ETCO2= end-tidal carbon dioxide; HR = heart rate; LMA = laryngeal mask airway; SpO2= oxygen saturation.
×
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
Fig. 2. Mean checklist and GRS scores over four simulation scenarios. Hollow markers represent checklist scores (left axis) and solid markers represent GRS scores (right axis). GRS = Global Rating Scale.
×
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
Fig. 3. Programming flowsheet for scenario 1. BP = blood pressure; ECG = electrocardiogram; HR = heart rate; LMA = laryngeal mask airway; min = minutes; OB = obstetrician; s = seconds; SPO2= oxygen saturation; VPB = ventricular premature beats.
×
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 4. Programming flowsheet for scenario 2. BP = blood pressure; ENT = Ear-nose-throat (specialist); HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 5. Programming flowsheet for scenario 3. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
Fig. 6. Programming flowsheet for scenario 4. BP = blood pressure; HR = heart rate; LMA = laryngeal mask airway; OB = obstetrician; s = seconds; SPO2= oxygen saturation.
×
Table 1. Percentage of Participants Completing the Critical Skills Checklist in Each Scenario
Image not available
Table 1. Percentage of Participants Completing the Critical Skills Checklist in Each Scenario
×
Table 2. Critical Skills Checklist Scores
Image not available
Table 2. Critical Skills Checklist Scores
×
Table 3. Global Rating Scale Scores
Image not available
Table 3. Global Rating Scale Scores
×
Table 4. Generalizability and Discrimination Analysis of Checklist and Global Rating Scale Scores Across Four Scenarios Rated by Three Raters
Image not available
Table 4. Generalizability and Discrimination Analysis of Checklist and Global Rating Scale Scores Across Four Scenarios Rated by Three Raters
×
Appendix 2-a. SIMULATION Study Survey Participant #: __________ Date: __________
Image not available
Appendix 2-a. SIMULATION Study Survey Participant #: __________ Date: __________
×
Appendix 2-b. SIMULATION Study Survey Participant #: __________ Date: __________
Image not available
Appendix 2-b. SIMULATION Study Survey Participant #: __________ Date: __________
×