Free
Perioperative Medicine  |   June 2019
Pediatric Risk Stratification Is Improved by Integrating Both Patient Comorbidities and Intrinsic Surgical Risk
Author Notes
  • From the Division of Cardiac Anesthesia (V.G.N., J.A.D.), Department of Anesthesiology, Critical Care and Pain Medicine, Boston Children’s Hospital, Harvard Medical School, Boston, Massachusetts (S.J.S., D.Z.); and the Division of Cardiac Anesthesia, Department of Anesthesia and Pain Medicine, The Hospital for Sick Children, University of Toronto, Toronto, Ontario Canada (D.F).
  • This article is featured in “This Month in Anesthesiology,” page 1A.
    This article is featured in “This Month in Anesthesiology,” page 1A.×
  • Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal’s Web site (www.anesthesiology.org).
    Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal’s Web site (www.anesthesiology.org).×
  • Submitted for publication September 11, 2018. Accepted for publication January 28, 2019.
    Submitted for publication September 11, 2018. Accepted for publication January 28, 2019.×
  • Address correspondence to Dr. Nasr: Boston Children’s Hospital, 300 Longwood Avenue, Boston, Massachusetts 02115. viviane.nasr@childrens.harvard.edu. This article may be accessed for personal use at no charge through the Journal Web site, www.anesthesiology.org.
Article Information
Perioperative Medicine / Clinical Science / Patient Safety / Pediatric Anesthesia
Perioperative Medicine   |   June 2019
Pediatric Risk Stratification Is Improved by Integrating Both Patient Comorbidities and Intrinsic Surgical Risk
Anesthesiology 6 2019, Vol.130, 971-980. doi:10.1097/ALN.0000000000002659
Anesthesiology 6 2019, Vol.130, 971-980. doi:10.1097/ALN.0000000000002659
Abstract

Editor’s Perspective:

What We Already Know about This Topic:

  • Risk stratification models to predict perioperative mortality in pediatric surgical populations are based on patient comorbidities, but do not take into consideration the intrinsic risk of the surgical procedures.

What This Article Tells Us That Is New:

  • Surgical procedures identified by specialty are not independent risk factors for perioperative mortality in pediatric patients. However, in multivariable predictive algorithms, the interaction of patient comorbidities with the intrinsic risk of the surgical procedure strongly predicts 30-day mortality.

Background: Recently developed risk stratification models for perioperative mortality incorporate patient comorbidities as predictors but fail to consider the intrinsic risk of surgical procedures. In this study, the authors used the American College of Surgeons National Surgical Quality Improvement Program Pediatric database to demonstrate the relationship between the intrinsic surgical risk and 30-day mortality and develop and validate an accessible risk stratification model that includes the surgical procedures in addition to the patient comorbidities and physical status.

Methods: A retrospective analysis of the American College of Surgeons National Surgical Quality Improvement Program Pediatric database was performed. The incidence of 30-day mortality was the primary outcome. Surgical Current Procedural Terminology codes with at least 25 occurrences were included. Multivariable logistic regression model was used to determine the predictors for mortality including patient comorbidities and intrinsic surgical risk. An internal validation using bootstrap resampling, and an external validation of the model were performed.

Results: The authors analyzed 367,065 surgical cases encompassing 659 unique Current Procedural Terminology codes with an incidence of overall 30-day mortality of 0.34%. Intrinsic risk of surgical procedures represented by Current Procedural Terminology risk quartiles instead of broad categorization was significantly associated with 30-day mortality (P < 0.001). Predicted risk of 30-day mortality ranges from 0% with no comorbidities to 4.7% when all comorbidities are present among low-risk surgical procedures and from 0.07 to 46.7% among high-risk surgical procedures. Using an external validation cohort of 110,474 observations, the multivariable predictive risk model displayed good calibration and excellent discrimination with area under curve (c-index) equals 0.95 (95% CI, 0.94 to 0.96; P < 0.001).

Conclusions: Understanding and accurately estimating perioperative risk by accounting for the intrinsic risk of surgical procedures and patient comorbidities will lead to a more comprehensive discussion between patients, families, and providers and could potentially be used to conduct cost analysis and allocate resources.

The global incidence of perioperative mortality in the pediatric surgical population is extremely low. However, the incidence of 30-day mortality can vary from 0.1 to 15% dependent on the patient’s comorbidities and physical status at the time of surgery.1–3  During the past decade, several groups have developed risk stratification models to improve prediction of perioperative major event (including death) in adults and enhance perioperative discussion of risk among physicians and the family, as well as improve resource allocation.4–7  The development of comparable risk stratification models has been undertaken in the pediatric surgical population as well.1,8 
In a recent study, we developed the Pediatric Risk Assessment score to predict perioperative mortality in neonates, infants, and children undergoing noncardiac surgery.1  The score includes patient’s age (e.g., less than 12 months), the presence of a neoplasm, the degree of emergency of the surgical procedure, the presence of at least one comorbidity (e.g., respiratory disease, congenital heart disease, kidney insufficiency, neurologic or hematologic disease), and characteristics of critical illness (e.g., mechanical ventilation, inotropic support, preoperative cardiopulmonary resuscitation). The score’s internal validation in a large cohort demonstrated an excellent accuracy in predicting perioperative mortality in children undergoing noncardiac surgery; however, the intrinsic risk of the surgical procedure was not included into our predictive model.
In adults, the intrinsic risk of surgical procedures for the occurrence of perioperative adverse cardiac events was recently stratified into three risk categories (e.g., low, intermediate, and high).6  In this study, the analysis demonstrated a wide variation in the intrinsic risk of particular surgical procedures despite the procedures being categorized within the same location (i.e., intrathoracic, intraperitoneal, urologic). To date, no such analysis of the intrinsic risk of individual pediatric surgical procedures has been published. The American College of Surgeons National Surgical Quality Improvement Program Pediatric Surgical Risk Calculator is a tool capable of estimating the risk of multiple complications and mortality for a wide variety of surgical procedures and concurrent patient comorbidities.2  However, the algorithm used by the American College of Surgeons National Surgical Quality Improvement Program Pediatric Surgical Risk Calculator to calculate the risk is invisible to the user and has never been publicly validated.
In this study, we used the American College of Surgeons National Surgical Quality Improvement Program Pediatric database to (1) demonstrate the relationship between the intrinsic surgical risk and 30-day mortality in neonates, infants, and children undergoing noncardiac surgery; and (2) develop and validate a risk stratification model that include patient comorbidities and physical status, as well as the surgical procedures after stratification for their intrinsic risk. Our objective is to develop an accessible risk stratification model.
Materials and Methods
Participating hospitals in the American College of Surgeons National Surgical Quality Improvement Program are not identified and Institutional Review Board approval was not required for this study. The data source and study population described below are similar to the development of the Pediatric Risk Assessment score.1 
Data Source
The American College of Surgeons National Surgical Quality Improvement Program Pediatric collects de-identified data on children less than 18 yr of age undergoing noncardiac surgery. It includes 129 variables, including preoperative risk factors, intraoperative characteristics, and 30-day postoperative mortality and morbidity outcomes in both the inpatient and outpatient settings.9  A site’s trained and certified Surgical Clinical Reviewer captures these data using a variety of methods including medical chart review. Adverse events and comorbidities reported in the database are determined by strict inclusion criteria. A systematic sampling strategy with an 8-day cycle is used to avoid bias in case selection and to ensure a diverse surgical case mix independent from the day of the week. In addition, to ensure the quality of the data collected, the American College of Surgeons National Surgical Quality Improvement Program Pediatric conducts inter-rater reliability audits of selected participating sites.10 
The results of the audits completed to date reveal an overall disagreement rate of approximately 2% for all assessed program variables. For the database, exclusion criteria included: patients 18 yr or older, trauma cases, solid organ transplantation, cardiac surgery, and cases coming from hospitals with an inter-rater reliability audit disagreement rate greater than 5%, or a 30-day follow-up rate less than 80%.
A total of 187 unique Current Procedural Terminology codes were excluded due to occurring less than 25 times, corresponding to 1,703 cases. Missing data were present on one or more of the variables used in the analysis in only 0.7% (2,513 of 369,176) of the cases in the 2012 to 2016 American College of Surgeons National Surgical Quality Improvement Program Pediatric database. Due to this extremely low rate of missing data, no missing data method was implemented.
Study Population
We performed a retrospective analysis of the 2012 to 2016 Pediatric databases of the American College of Surgeons National Surgical Quality Improvement Program database. The primary outcome variable for our analysis was the incidence of 30-day mortality. Current Procedural Terminology codes with fewer than 25 occurrences were excluded.11 
Variables
The following characteristics were considered: age, body weight, height, gender, American Society of Anesthesiologists (ASA) Physical Status classification, prematurity (fewer than 24, 24 to 36, and more than 36 weeks of gestation), type of procedure (elective vs. urgent surgery), preoperative respiratory disease (e.g., asthma, chronic lung or airway diseases, cystic fibrosis), preoperative oxygen supplementation, tracheostomy, liver and pancreatic diseases, diabetes, congenital heart disease, acute or chronic kidney disease, neurologic disease (e.g., mental retardation, cerebral palsy, central nervous system disease, intracerebral hemorrhage, seizure), immune disease, preoperative use of steroids, neoplasm, chemotherapy, preoperative inotropic support, preoperative mechanical ventilation, preoperative cardiopulmonary resuscitation, and preoperative transfusion (defined as transfusion of whole blood or erythrocytes during the 48 h before surgery).
Surgical type was categorized based on Current Procedural Terminology codes. Intrinsic surgical risk was determined by utilizing surgery Current Procedural Terminology codes with at least 25 occurrences in the sample (659 unique Current Procedural Terminology codes). Current Procedural Terminology risk quartiles were built utilizing the empirical 30-day mortality rates for each Current Procedural Terminology code and creating four groups of Current Procedural Terminology codes corresponding to increasing intrinsic surgical risk. The range for 30-day mortality rate for Current Procedural Terminology risk quartile 1 was 0%, risk quartile 2 was greater than 0% to less than 0.14%, risk quartile 3 was greater than or equal to 0.14% to less than 1.15%, and risk quartile 4 was greater than or equal to 1.15%. Current Procedural Terminology risk quartiles 1 and 2 comprised the low-risk procedure category, and quartiles 3 and 4 the high-risk procedures. The cut-offs of 30-day mortality rate to define the Current Procedural Terminology risk quartiles were determined by examining the distribution on the case level of mortality rates based on Current Procedural Terminology codes. Since the 30-day mortality rate in the National Surgical Quality Improvement Program database is very low, the four risk quartiles are not equal in size in terms of number of surgical cases.
Statistical Analysis
Comorbidity and case complexity data are presented as median and interquartile range and number and percentage for categorical data. Univariate statistical testing was done using Wilcoxon rank sum tests and chi-square tests, as appropriate. Multivariable logistic regression modeling building using stepwise backward elimination with removal criteria of P > 0.05 was applied to identify independent predictors of 30-day mortality and to develop a multivariable algorithm combining both patient comorbidities and intrinsic surgical risk to predict the risk of 30-day mortality.12 
Patient comorbidities and intrinsic surgical risk were considered as predictors of 30-day mortality. Using the likelihood ratio test to assess significance, five variables were included in the final model: body weight (kg), ASA Physical Status classification, preoperative sepsis, preoperative inotropic support, and preoperative ventilator dependence (all within 24 h before surgery). For the risk algorithm, body weight was dichotomized to less than or greater than 5 kg, and ASA Physical Status classification was dichotomized to create a binary indicator of high ASA (ASA Physical Status III or higher). The dichotomization of continuous risk factors was based on clinical experience and the ability to provide high sensitivity and specificity for discrimination of cases with and without 30-day mortality.
Dichotomous patient comorbidities and intrinsic surgical risk variables were utilized in the full multivariable predictive algorithm of 30-day mortality. A simplified multivariable predictive algorithm was created considering the number of comorbidity risk factors present.
Multivariable logistic regression results are presented as adjusted odds ratios, 95% CIs, and P values. The predictive algorithms for the probability of 30-day mortality are presented as empirical probabilities, model-based predicted probabilities, and 95% CIs as a measure of precision of the model-based estimates, stratified by intrinsic surgical risk.
A two-tailed α level of 0.05 was used as the threshold for statistical significance. Stata 15.0 was utilized for all statistical analyses (StataCorp, USA).
Model Validation
Internal validation was performed for our final multivariable model utilizing 500 bootstrap resamples.13,14  In our internal bootstrap validation, we assessed model performance using the c-index (area under the curve), the bias-corrected Somers D rank correlation, Nagelkerke R2, the slope and intercept of the logistic calibration equation, the maximum absolute difference in predicted and calibrated probabilities (Emax), the discrimination index D, the unreliability index U, and the Brier quadratic probability score B. Internal bootstrap validation was performed by re-fitting the multivariable model in 500 bootstrap resample dataset with replacement produced by the 2012 to 2016 National Surgical Quality Improvement Program Pediatric database.
Furthermore, external model validation was performed using the 2017 National Surgical Quality Improvement Program Pediatric database in order to assess the generalizability of our model in an external cohort with a similar patient mix. Model calibration was assessed using the Hosmer–Lemeshow goodness-of-fit test, where a nonsignificant P value indicates that the prognostic multivariable model generalizes well to the new cohort. Model discrimination was assessed using the area under the receiving operating characteristic curve. An area under the receiver operating characteristic curve of 0.800 to 0.899 will be considered to demonstrate acceptable model discrimination, and values greater than or equal to 0.900 will be considered outstanding model discrimination.12  Observed probabilities of 30-day mortality were compared to the fitted probabilities produced in our multivariable algorithm using this external cohort.
Power Analysis and Sample Size Considerations
The sample sizes that were analyzed in this study based on 2012 to 2016 from the American College of Surgeons National Surgical Quality Improvement Program Pediatric database among patients with at least three of the identified comorbidities provide more than 90% statistical power for detecting a difference in 30-day mortality between low and high intrinsic surgical risk procedures of 10%, based on logistic regression modeling. Power analyses were performed using nQuery Advisor 8.2.2 (Statistical Solutions Ltd., Ireland).
Results
A final sample of 367,065 surgical cases encompassing 659 unique Current Procedural Terminology codes was obtained for analysis. All Current Procedural Terminology codes with fewer than 25 occurrences were excluded. Among these cases, 1,252 (0.34%) involved 30-day mortality. All Current Procedural Terminology codes were categorized into four intrinsic risk quartiles. The complete list of all procedures is listed in the appendix.
Nonsurvivors were more often neonates (41.8% vs. 4.2%), had low body weight (less than 5 kg; 65% vs. 9%), had higher ASA Physical Status classification greater than III (96.7% vs. 25.5%), higher rates of preoperative sepsis (30.8% vs. 7.9%), inotropic support (32.4% vs. 0.6%), congenital heart disease (50.2% vs. 10.0%), and ventilator dependence within 48 h before surgery (65.5% vs. 2.9%; all P < 0.001). Intrinsic risk of surgical procedures represented by Current Procedural Terminology risk quartiles was significantly associated with 30-day mortality (P < 0.001; table 1). Multivariable logistic regression analysis revealed the following factors as being independent predictors of 30-day mortality: weight, ASA Physical Status classification, preoperative sepsis, inotropic support, ventilator dependence, and risk quartile. Using stepwise backward elimination with removal criteria of P > 0.05, neonatal status, sex, and congenital heart disease were eliminated. Table 2 displays the adjusted model of 30-day mortality based on all comorbidity and case complexity risk factors.
Table 1.
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Univariate Analysis of Comorbidity and Complexity Patient Characteristics×
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Table 1.
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Univariate Analysis of Comorbidity and Complexity Patient Characteristics×
×
Table 2.
Multivariable Model for 30-Day Mortality
Multivariable Model for 30-Day Mortality×
Multivariable Model for 30-Day Mortality
Table 2.
Multivariable Model for 30-Day Mortality
Multivariable Model for 30-Day Mortality×
×
The independent predictors were multiplexed to create a multivariable predictive algorithm for the risk of 30-day mortality. A predicted probability of 30-day mortality was calculated for all covariate patterns of comorbidities and stratified by intrinsic surgical risk. Supplemental Digital Content 1 (http://links.lww.com/ALN/B891) displays the full risk algorithm, with empirical mortality rates as well as model-based mortality probability and 95% CIs. Among low-risk surgical procedures, the risk of 30-day mortality ranged from 0.00% (95% CI, 0.00 to 0.01%) when no comorbidities are present, to 4.74% (95% CI, 3.17 to 7.03%) when all comorbidities are present. This association is magnified among high-risk surgical procedures, where the risk of 30-day mortality ranged from 0.07% (95% CI, 0.05 to 0.09%) when no comorbidities are present, to 46.72% (95% CI, 43.04 to 50.44%) when all comorbidities are present.
The bootstrapped results of the internal validation suggest that our predictive model has an excellent internal validity and model performance with a c-index or area under the curve of 0.961, and a bias-corrected Somers D rank correlation of 0.922. The Nagelkerke R2 measure was 0.395. The intercept and slope of an overall logistic calibration equation were 0.020 and 1.007. The maximum absolute difference in predicted and calibrated probabilities, or Emax, was 0.006. The discrimination index D was 0.018 and the unreliability index U was 0, resulting in an overall quality index or logarithmic probability score Q equals 0.018. The Brier quadratic probability score B was 0.003.
In addition to an internal bootstrap validation, we performed an external model validation using the 2017 National Surgical Quality Improvement Program Pediatric database. The results of the observed and the fitted (expected) probabilities of 30-day mortality are found in table 3. Using this external validation cohort of 110,474 observations, our multivariable predictive risk model displayed good calibration to the data (Hosmer–Lemeshow goodness-of-fit P = 0.116) and outstanding model discrimination (area under curve [c-index] equals 0.953; 95% CI, 0.944 to 0.961; P < 0.001).
Table 3.
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database×
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
Table 3.
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database×
×
A simplified algorithm was created using the number of comorbidity risk factors rather than the specified combinations. The results of this model are found in Supplemental Digital Content 2 (http://links.lww.com/ALN/B892). In this simplified algorithm the interaction between comorbidities and case complexity remains. Within the high-risk surgical procedure category, the risk of mortality is exacerbated. The association between Current Procedural Terminology risk quartile and risk of mortality is modified by patient comorbidity profile (fig. 1). In this simplified model, the risk of 30-day mortality ranges from 0 to 5.2% among low-risk procedures, whereas it ranges from 0.05 to 39.3% among high-risk procedures.
Fig. 1.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
Fig. 1.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
×
Discussion
This study demonstrates the relationship between the intrinsic surgical risk and 30-day mortality for 659 specific pediatric surgical procedures. Consequently, procedures typically characterized by procedure location (i.e., intrathoracic, intraperitoneal) or surgical specialty (i.e., plastics, urology) are now grouped by intrinsic risk. When surgical procedures are identified by specialty, the relationship between mortality and a specific procedure is not possible. In fact, pediatric postoperative mortality has been shown to be caused primarily by patient- and anesthesia-related factors when the surgical procedures are grouped by specialty.15  The granularity is important because when intrinsic operative risk is analyzed in conjunction with patient comorbidities it becomes clear that the interaction of these two variables strongly predicts perioperative mortality. As demonstrated in figure 1, procedures with low intrinsic risk can be performed on children with three or fewer concurrent comorbidities with low mortality with a steady increase in mortality seen when four and five comorbidities are present. In contrast, an exponential increase in mortality associated with each additional comorbidity above two was observed for procedures with high intrinsic risk. Specifically, when no comorbidities are present, the probability of 30-day mortality is similar between the low and high intrinsic surgical risk groups (0% vs. 0.05%). In contrast, when all five comorbidities are present, the probability of 30-day mortality is much lower in the low intrinsic surgical risk compared to the high intrinsic surgical risk group (5.2% vs. 39.3%).
Analysis of 3.7 million adult patients between 1991 and 2005 suggested that the most robust predictor of postoperative mortality should be a model containing patient demographics, comorbidities, and surgical procedures categorized by anatomic location into 36 subcategories. There was a 256-fold difference in mortality between the lowest (nucleus pulposus surgery) and highest (liver transplant) risk surgical procedures.16  A recent investigation in adults further expanded on the concept of intrinsic surgical risk by analyzing 1,880 Current Procedural Terminology codes to categorize 202 specific surgical procedures into low, intermediate, and high intrinsic risk. Intrinsic risk, thus determined, proved to be a more robust predictor of perioperative adverse cardiac events than surgical procedures grouped by anatomic location.6 
The physiologic responses initiated in the cardiovascular, pulmonary, endocrine, coagulation, and immune systems by direct surgical tissue injury in addition to the responses initiated by mechanical deformation of organs, blood loss, core temperature variations, and fluid shifts vary tremendously depending on the invasiveness and the duration of the surgical procedure. The greater the physiologic response to a surgical procedure, the greater is the intrinsic surgical risk. This is consistent with the finding that the physiologic compromise and intrinsic surgical risk associated with an open colectomy is greater than that of excision of a skin lesion and with the finding that procedures in the same body cavity and on the same organ (partial splenectomy, risk quartile 1 and total splenectomy, risk quartile 4) would be associated with substantially different intrinsic risk dependent on the complexity of the procedure.
It is interesting that the presence of congenital heart disease did not warrant retention in our multivariable model in light of previous work demonstrating that in children undergoing noncardiac surgery major and severe congenital heart disease, as defined by functional status and residual lesion burden, is associated with increased mortality.8,17  This is likely due to the fact that the presence of congenital heart disease was considered as a binary variable in this analysis and the severity of congenital heart disease was not considered. It is also likely that children with major and severe congenital heart disease underwent procedures with high intrinsic risk less frequently than children without major or severe congenital heart disease.
During the informed consent process, parents are interested in receiving comprehensive information regarding their child’s surgical procedure including delineation of possible complications and provision of this additional information does not increase parental anxiety.18  Use of this simple and easily applicable risk categorization will provide parents with a more comprehensive overview of the risk associated with a particular surgical procedure during the informed consent process. In addition, categorization of the risk based on specific surgical procedures and patient comorbidities has the potential to improve preoperative optimization and allocation of resources.19 
This study has several strengths and limitations. The limitations include the retrospective nature of the study design. The use of a large multi-institutional database may include missing data, miscoded diagnoses, or procedures. However, the American College of Surgeons National Surgical Quality Improvement Program is well designed and undergoes a thorough audit that makes it more accurate and informative than other administrative databases. While the American College of Surgeons National Surgical Quality Improvement Program database provides some granularity as regards the severity of comorbidities it does not provide the type of detailed information (e.g., creatinine clearance, pulmonary function tests, blood gas analysis) necessary to further sub-categorize disease severity. In addition, because the American College of Surgeons National Surgical Quality Improvement Program database does not contain geographic or site-specific identification it was impossible to analyze the impact of hospital setting and anesthesia provider on outcome as was done in the European Anaesthesia PRactice In Children Observational Trial (APRICOT) study.20  Assessing generalizability in an external cohort with a similar patient case mix is important to assess model performance.14  A major strength of this study is the external validation with a separate 2017 National Surgical Quality Improvement Program cohort that revealed very strong model performance and generalizability. Demonstrating generalizability of the predictive algorithm in institutions remains needed to confirm utility in clinical practice.
In conclusion, this study demonstrates that the combination of intrinsic surgical risk and patient comorbidities accurately estimates the risk of 30-day mortality in children and allows stratification of this risk. High-intrinsic surgical risk in children contributes significantly to 30-day mortality across the full range of patient comorbidities and is particularly impactful in patients presenting with several of the five identified comorbidities.
Acknowledgments
The authors thank the American College of Surgeons National Surgical Quality Improvement Program and the hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program, the source of data used herein. The American College of Surgeons National Surgical Quality Improvement Program has not verified, and is not responsible for, the statistical validity of the data analysis or the conclusions derived by the authors.
Research Support
This study was solely supported by the Department of Anesthesiology, Critical Care and Pain Medicine at Boston Children’s Hospital, Harvard Medical School, Boston, Massachusetts.
Competing Interests
The authors declare no competing interests.
References
Nasr, VG, DiNardo, JA, Faraoni, D : Development of a pediatric risk assessment score to predict perioperative mortality in children undergoing noncardiac surgery. Anesth Analg. 2017; 124:1514–9. [Article] [PubMed]
Kraemer, K, Cohen, ME, Liu, Y, Barnhart, DC, Rangel, SJ, Saito, JM, Bilimoria, KY, Ko, CY, Hall, BL : Development and evaluation of the American College of Surgeons NSQIP Pediatric Surgical Risk Calculator. J Am Coll Surg. 2016; 223:685–93. [Article] [PubMed]
Bonasso, PC, Dassigner, MS, Ryan, ML, Gowen, MS, Burford, JM, Smith, SD : 24-hour and 30-day perioperative mortality in pediatric surgery. J Pediatr Surg. 2019; 54:628–30. [Article] [PubMed]
Le Manach, Y, Collins, G, Rodseth, R, Le Bihan-Benjamin, C, Biccard, B, Riou, B, Devereaux, PJ, Landais, P : Preoperative score to predict postoperative mortality (POSPOM): Derivation and validation. Anesthesiology. 2016; 124:570–9. [Article] [PubMed]
Moonesinghe, SR, Mythen, MG, Das, P, Rowan, KM, Grocott, MP : Risk stratification tools for predicting morbidity and mortality in adult patients undergoing major surgery: Qualitative systematic review. Anesthesiology. 2013; 119:959–81. [Article] [PubMed]
Liu, JB, Liu, Y, Cohen, ME, Ko, CY, Sweitzer, BJ : Defining the intrinsic cardiac risks of operations to improve preoperative cardiac risk assessments. Anesthesiology. 2018; 128:283–92. [Article] [PubMed]
Stefani, LC, Gutierrez, CS, Castro, SMJ, Zimmer, RL, Diehl, FP, Meyer, LE, Caumo, W : Derivation and validation of a preoperative risk model for postoperative mortality (SAMPE model): An approach to care stratification. PLoS One. 2017; 12:e0187122. [Article] [PubMed]
Faraoni, D, Vo, D, Nasr, VG, DiNardo, JA : Development and validation of a risk stratification score for children with congenital heart disease undergoing noncardiac surgery. Anesth Analg. 2016; 123:824–30. [Article] [PubMed]
User Guide for the 2014 ACS NSQIP Pediatric Participant Use Data File. Available at: http://www.pediatric.acsnsqip.org/acsNsqipDataPed/jsp/pdf/PEDS.ACS.NSQIP.PUF.UserGuide.2014.pdf. Accessed August 2018.
Sellers, MM, Reinke, CE, Kreider, S, Meise, C, Nelis, K, Volpe, A, Anzlovar, N, Ko, C, Kelz, RR : American College of Surgeons NSQIP: Quality in-training initiative pilot study. J Am Coll Surg. 2013; 217:827–32. [Article] [PubMed]
Bilimoria, KY, Liu, Y, Paruch, JL, Zhou, L, Kmiecik, TE, Ko, CY, Cohen, ME : Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013; 217:833–42.e13. [Article] [PubMed]
Hosmer, DW, Lemeshow, S, Sturdivant, RX : Applied logistic regression. 20133rd ed. Hoboken, NJ: John Wiley & Sons.
Steyerberg, EW, Harrell, FEJr : Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol. 2016; 69:245–7. [Article] [PubMed]
Altman, DG, Vergouwe, Y, Royston, P, Moons, KG : Prognosis and prognostic research: Validating a prognostic model. BMJ. 2009; 338:b605. [Article] [PubMed]
van der Griend, BF, Lister, NA, McKenzie, IM, Martin, N, Ragg, PG, Sheppard, SJ, Davidson, AJ : Postoperative mortality in children after 101,885 anesthetics at a tertiary pediatric hospital. Anesth Analg. 2011; 112:1440–7. [Article] [PubMed]
Noordzij, PG, Poldermans, D, Schouten, O, Bax, JJ, Schreiner, FA, Boersma, E : Postoperative mortality in The Netherlands: A population-based analysis of surgery-specific risk in adults. Anesthesiology. 2010; 112:1105–15. [Article] [PubMed]
Faraoni, D, Zurakowski, D, Vo, D, Goobie, SM, Yuki, K, Brown, ML, DiNardo, JA : Post-operative outcomes in children with and without congenital heart disease undergoing noncardiac surgery. J Am Coll Cardiol. 2016; 67:793–801. [Article] [PubMed]
Kain, ZN, Wang, SM, Caramico, LA, Hofstadter, M, Mayes, LC : Parental desire for perioperative information and informed consent: A two-phase study. Anesth Analg. 1997; 84:299–306. [Article] [PubMed]
Snowden, CP, Anderson, H : Preoperative optimization: Rationale and process: Is it economic sense? Curr Opin Anaesthesiol. 2012; 25:210–6. [Article] [PubMed]
Habre, W, Disma, N, Virag, K, Becke, K, Hansen, TG, Jöhr, M, Leva, B, Morton, NS, Vermeulen, PM, Zielinska, M, Boda, K, Veyckemans, F ; APRICOT Group of the European Society of Anaesthesiology Clinical Trial Network: Incidence of severe critical events in paediatric anaesthesia (APRICOT): A prospective multicentre observational study in 261 hospitals in Europe. Lancet Respir Med. 2017; 5:412–25. [Article] [PubMed]
Appendix
This Appendix describes the procedures in each risk quartile (RQ) with 87 procedures in RQ1, 15 in RQ2, 46 in RQ3, and 33 in RQ4.
Risk Quartile 1
  1. Anterior neck procedures (thyroid/thyroglossal duct)

  2. Arthroscopy

  3. Arthrotomy (knee)

  4. Arteriovenous malformation supratentorial

  5. Branchial cleft

  6. Bullae resection

  7. Open cholecystectomy

  8. Clitoroplasty

  9. Colotomy/duodenotomy/foreign body

  10. Craniosynostosis

  11. Craniotomy: bone flap/cyst

  12. Cystoscopy and ureteroscopy

  13. Digit reconstruction

  14. Dislocation of hip and femur

  15. Drainage of neck abscess

  16. Ear procedures (tympanoplasty, mastoidectomy, others)

  17. Epiphyseal arrest

  18. External fixation (bone)

  19. Excision of parotid tumor/gland

  20. Facial bone reconstruction

  21. Foot division of joint capsule, ligament, or cartilage

  22. Forehead reconstruction

  23. Fracture/dislocation of humerus/tibia/foot

  24. Hypospadias

  25. Incision and drainage of submandibular/submental gland

  26. Implantation/revision/repositioning of tunneled intrathecal or epidural

  27. Joint procedure

  28. Laminectomy (with or without neoplasm)

  29. Laparoscopic colectomy

  30. Laparoscopic ileostomy

  31. Laparoscopic jejunostomy

  32. Laparoscopic hernia

  33. Laparoscopic cyst aspiration

  34. Laparoscopic enterolysis

  35. Laparoscopic nephrectomy

  36. Laryngoplasty/laryngoplasty cricoid

  37. Lithotripsy

  38. Lower gastrointestinal procedures/fistula/anoplasty

  39. Lymphadenectomy (except deep cervical)

  40. Mastectomy

  41. Mediastinal tumor resection

  42. Nephrectomy

  43. Neuroendoscopic replacement of ventricular catheter

  44. Oophorectomy

  45. Orchidopexy

  46. Osteoplasty

  47. Osteotomy (limb) excluding hip osteotomy

  48. Ovarian cyst drainage/resection

  49. Palatoplasty, secondary repair of cleft palate/lip

  50. Partial excision of bone tumor

  51. Partial splenectomy

  52. Partial colectomy

  53. Percutaneous nephrostolithotomy

  54. Pharyngoplasty

  55. Procedure on tendons and/or muscles

  56. Procedures related to the bile duct

  57. Replacement of cranial nerve stimulator

  58. Reconstruction pectus excavatum

  59. Rectal procedure

  60. Renal biopsy

  61. Renal procedures

  62. Repair of syndactyly

  63. Retropharyngeal/peritonsillar abscess

  64. Revision colostomy/ileostomy

  65. Rhinoplasty with/without revision to nasal tip

  66. Salpingectomy

  67. Simple diaphragm repair

  68. Sinus endoscopy (partial ethmoidectomy)

  69. Sinus endoscopy: sphenoidotomy

  70. Skin graft

  71. Skin lesion excision

  72. Sympathectomy

  73. Sleeve gastrectomy

  74. Enterocystoplasty

  75. Spine fusion reinsertion or removal, exploration

  76. Subarachnoid/subdural shunt

  77. Subdural implantation of electrodes

  78. Thoracoscopic thymus resection

  79. Tracheoplasty

  80. Urinary tract procedures

  81. Upper gastrointestinal procedure

  82. Ureteral catheterization

  83. Urethral procedures

  84. Vaginoplasty

  85. Varicocele excision or ligation

  86. Varicose

  87. Ventral hernia (omphalocele)

Risk Quartile 2
  1. Appendectomy

  2. Craniotomy: bone tumor resection

  3. Gastrostomy closure

  4. Hemiepiphyseal arrest

  5. Laminectomy with release of spinal cord

  6. Lap splenectomy

  7. Laparoscopic cholecystectomy

  8. Lymphadenectomy (deep cervical)

  9. Neuro: Implantation of cranial nerve neurostimulator

  10. Orchidopexy (abdominal approach)

  11. Osteotomy (hip)

  12. Primary plastic cleft lip/palate

  13. Pyloromyotomy

  14. Sinus surgery: ethmoidectomy

  15. Sinus surgery: maxillary antrostomy

Risk Quartile 3
  1. Arthrodesis

  2. Arthrotomy (hip infection)

  3. Brain tumor resection/open or endoscopy

  4. Bronchoscopy (foreign body removal)

  5. Colectomy for congenital megacolon

  6. Craniectomy with cervical laminectomy

  7. Craniotomy: electrode placement for seizure monitoring

  8. Cystoscopy and ureteroscopy with stent placement

  9. Cystostomy with drainage

  10. Diagnostic thoracoscopy (mediastinum)

  11. Diverticulum

  12. Enterostomy closure

  13. Enterotomy/foreign body

  14. Enterolysis

  15. Excision of submandibular gland

  16. Exploratory retroperitoneal

  17. Fracture of femoral shaft

  18. Gastrostomy/foreign body

  19. Implantation or replacement of drug infusion device

  20. Intussusception

  21. Laparoscopic colostomy or cecostomy

  22. Laparoscopic esophageal procedure

  23. Laparoscopic gastrostomy

  24. Laparoscopic small intestine resection

  25. Laparoscopy/neoplasm related

  26. Laparoscopic proctectomy with pull-through

  27. Laparoscopic proctectomy and colectomy

  28. Large omphalocele/final reduction

  29. Laryngoscopy with operative procedure

  30. Mammaplasty

  31. Myelomeningocele

  32. Nephrectomy with rib resection, ureterectomy

  33. Osteotomy (hip) with fixation

  34. Placement of enterostomy/rev of complicated enterostomy

  35. Pleurodesis (thoracoscopy)

  36. Pulmonary decortication

  37. Pulmonary wedge resection

  38. Repair of low imperforate anus

  39. Replacement or revision ventriculoperitoneal shunt/ventriculocisternostomy

  40. Rhinoplasty including any of the following septal repair/choanal/polyp removal/sinus endoscopy

  41. Salpingo-oophorectomy

  42. Small intestine resection (no tapering)

  43. Small omphalocele with primary closure

  44. Thoracotomy for lobectomy/pneumonectomy/segmentectomy/wedge resection

  45. Transplantation of ureter to skin

  46. Vesicostomy

Risk Quartile 4
  1. Burr holes for implanting ventricular catheter, cerebral electrodes

  2. Colectomy

  3. Complicated nephrectomy from prior surgery

  4. Creation of ventriculoperitoneal, atrial, jugular, or others shunt

  5. Diagnostic thoracoscopy (mediastinal/pericardial)

  6. Exploratory laparotomy (neoplasm)

  7. Gastric bypass Roux-en-Y

  8. Hartmann procedure colectomy

  9. Hepatectomy/hepatic lobectomy

  10. Ileostomy

  11. Imbrication of diaphragm for eventration

  12. Intraperitoneal catheter for dialysis

  13. Laparoscopy/intraperitoneal catheter

  14. Large omphalocele

  15. Liver wedge biopsy

  16. Malrotation correction and/or reduction of midgut volvulus.

  17. Mediastinotomy/foreign body removal

  18. Open colostomy or cecostomy/gastrostomy

  19. Pancreatectomy

  20. Paraesophageal/diaphragmatic hernia

  21. Parietal pleurectomy (thoracoscopic)

  22. Peritoneal abscess drainage

  23. Sinus surgery: sphenoidectomy

  24. Small intestine resection (with tapering)

  25. Suture for perforated ulcer/wound/injury to the gastrointestinal tract

  26. Thoracic approach for esophageal surgery

  27. Thoracoscopic with foreign body removal (intrapleural)

  28. Thoracoscopy/thoracotomy for lung biopsy nodule/mass/infiltrate/cyst removal

  29. Thyroidectomy

  30. Total splenectomy

  31. Tracheal stenosis resection

  32. Tracheoscopy and laryngoscopy with biopsy/newborn

  33. Tracheostomy

Fig. 1.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
Fig. 1.
Probability of 30-day mortality based on multivariable modeling is modified by the number of significant comorbidity risk factors, ranging from none (0) to 5. The comorbidities are low body weight (less than 5 kg), American Society of Anesthesiologists Physical Status III or higher, preoperative sepsis, inotropic support, and ventilator dependence. The impact of intrinsic surgical complexity risk on 30-day mortality is magnified among patients with a greater number of the five comorbidities.
×
Table 1.
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Univariate Analysis of Comorbidity and Complexity Patient Characteristics×
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Table 1.
Univariate Analysis of Comorbidity and Complexity Patient Characteristics
Univariate Analysis of Comorbidity and Complexity Patient Characteristics×
×
Table 2.
Multivariable Model for 30-Day Mortality
Multivariable Model for 30-Day Mortality×
Multivariable Model for 30-Day Mortality
Table 2.
Multivariable Model for 30-Day Mortality
Multivariable Model for 30-Day Mortality×
×
Table 3.
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database×
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
Table 3.
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database
External Validation of Simplified Multivariable Predictive Algorithm using the 2017 NSQIP Pediatric Database×
×