Correspondence  |   March 2008
Explanatory versus  Pragmatic Trials? The Methods Make the Difference
Author Affiliations & Notes
  • Paul S. Myles, M.B., B.S., M.P.H., M.D., F.C.A.R.C.S.I., F.A.N.Z.C.A., F.R.C.A.
  • †Alfred Hospital and Monash University, Melbourne, Victoria, Australia.
Article Information
Correspondence   |   March 2008
Explanatory versus  Pragmatic Trials? The Methods Make the Difference
Anesthesiology 3 2008, Vol.108, 543-544. doi:10.1097/ALN.0b013e3181654728
Anesthesiology 3 2008, Vol.108, 543-544. doi:10.1097/ALN.0b013e3181654728
In Reply:—
We thank the correspondents for their interest in our study1 and would like to respond to the various points of view raised.
Drs. Mirski and Gottschalk question the lack of the ability of nitrous oxide to reduce early recovery times in our trial. In the ENIGMA trial, anesthesiologists were instructed to adjust the depth of anesthesia to an appropriate level using clinical signs and/or electroencephalographic monitoring, aiming to reflect routine practice. No specific targets for depth of anesthesia were set. The fact that times to eye opening were similar in the two groups is an indication that the anesthesiologists were successful in maintaining equivalent depths of anesthesia with a variety of anesthetic combinations. Specifically, our results show that comparable depths of anesthesia and emergence times can be achieved with or without nitrous oxide. The emergence times are at least as good as in most other studies reporting recovery profiles after (median) 4-h surgeries.
Numerous studies have confirmed elevated plasma homocysteine concentration after nitrous oxide exposure. This follows inhibition of methionine synthase and occurs even at low concentrations (<20%). Currently, we do not know whether this biochemical change leads to cardiovascular morbidity, but it is a likely explanation for the increased rates of myocardial ischemia reported by others.2 We believe a discussion of the trend toward more frequent adverse cardiac outcomes was warranted, in light of the cogent physiologic basis for concerns regarding this important outcome. Given the significant costs of cardiovascular complications to patients and the community, we believe that these findings demand further study. We have thus embarked on a follow-up study in 7,000 patients to test this hypothesis.1
The controversy over the potentially detrimental effects of nitrous oxide in neurosurgery has raged for decades, with ongoing divergence of opinion and conflicting evidence. We outlined some of the criticisms that had been levelled at nitrous oxide, in addition to highlighting its possible benefits. The only neurologic outcome we measured was stroke, and the incidence of this complication was too low for us to draw any conclusions.
Meta-analyses show that omission of nitrous oxide reduces the risk of postoperative nausea and vomiting (PONV) regardless of whether propofol or volatile anesthetics are used to maintain anesthesia.3,4 Our study found a strong adverse effect of nitrous oxide across a range of patient and surgical types with respect to severe PONV, the definition of which was based on an extended period of symptoms or failure of therapy. This effect was evident despite the majority of patients in both arms of our study receiving a volatile anesthetic, higher concentrations of which were used in the nitrous oxide–free arm. Drs. Mirski and Gottschalk misrepresent published consensus guidelines, in that an expected PONV incidence of 10% is a low-risk setting and does not justify PONV prophylaxis. We agree that prophylactic antiemetics are efficacious in moderate- and high-risk settings, and they were administered to 35% of patients in our ENIGMA trial. In any case, the adverse effect of nitrous oxide on PONV was apparent whether or not prophylactic antiemetics were used.
We chose hospital stay as our primary endpoint because we were uncertain about which adverse events would predominate but expected that any of these could affect duration of stay. The effect was borderline (P  = 0.06), but if true, a 9% increased rate of delayed discharge is clinically important when applied to millions of patients every year. Intensive care stay was prolonged, reflecting more serious complications in patients exposed to nitrous oxide.
The secondary analyses of postoperative complications (results given in table 3) were prespecified comparisons of randomized groups (nitrous oxide–based vs.  nitrous oxide–free anesthesia) according to the intention-to-treat principle. In addition, these analyses were adjusted for potential confounding variables. Our conclusions were based on the results of these analyses. The subgroup analyses (results given in fig. 4) were post hoc  and not controlled and were presented so readers could see the trend in results across a range of clinical subgroups of possible interest.
We thank Drs. Dawson and Hardman for their comments, which are relevant to the issues surrounding the conduct of large perioperative trials. Tight protocol control of all of the numerous variables surrounding modern anesthetic and surgical management is not practicable in large multicenter randomized trials,5,6 nor is it desirable.7,8 Construction of narrow protocols inevitably leads to criticism of the results on the basis that “we do things differently here.” In contrast, flexibility in the wider aspects of patient management is more likely to provide answers that broadly reflect common practice and can be more readily generalized.8 
Very large trials such as ours make the risks of asymmetry between groups in these variables (e.g.  , antibiotic and antiemetic use) much lower than is the case in smaller single-center studies. Large trials balance known and unknown confounding factors. The exception to this, of course, is in variables that are directly affected by the intervention being tested. An example of this is the difference in cumulative minimum alveolar concentration scores between the two groups, as pointed out by Drs. Dawson and Hardman. Given that the study protocol stipulated a standard clinical approach to maintaining and monitoring depth of anesthesia, far from invalidating our results, it is likely to reflect one of many real differences between modern approaches to the conduct of nitrous oxide–based and nitrous oxide–free anesthesia, which may impact on outcome. With regard to the greater use of propofol where nitrous oxide was not used, we would make a similar point.
Retention of nitrogen in the inspired gas mixture is a further example of the anesthetic regimen being modified by avoidance of nitrous oxide. We agree that, on theoretical grounds, retention of some nitrogen in the inspired mixture, as stipulated by our protocol, may well have contributed to better pulmonary outcomes in the nitrous oxide–free group, by reducing atelectasis. Some anesthesiologists have a mistaken belief that nitrous oxide provides protection against absorption atelectasis, but this is not the case.9 
Dr. White et al.  seem to overlook two of the key design features of large randomized trials: (1) The large sample size provides balance of the numerous possible confounding factors that could affect the outcomes of interest, and (2) the inclusion of a variety of practice settings, with varying anesthetic and surgical techniques, represents “real-world” anesthesia and thus provides comfort to those concerned with whether the study results apply to them. Others have expanded on these issues extensively.5,6,8 We provided details of risk factors for PONV and PONV risk scores in our table 1. Furthermore, the large sample size provides opportunity to test for variability of effect according to specific factors; we provided results of such analyses in figure 4.
Small single-center trials suffer from restrictive regimens that may not represent typical practice, and they are often underpowered to detect important differences in outcome. For example, one of the studies cited by Dr. White et al.  reported a power calculation to detect a difference in PONV rates of 40% and 30%, and arrived at 35 patients per group,10 whereas the true value is 450 patients per group. Underpowered studies abound in anesthetic journals. They are used by some to support a point of view, but such views conveniently ignore a larger body of relevant evidence with contrary findings—a few small trials do not replace well-conducted meta-analyses of all relevant trials.3,4 In any case, the ambulatory care setting was not included in our study population, and we have not made any conclusions in this regard.
Drs. Sharma and Dash suggested that nitrous oxide is a useful adjunct to anesthesia because it is “inexpensive, widely available, and safe.” Certainly, ongoing widespread use around the world mandates outcomes research on the effectiveness and safety of nitrous oxide. However, this view does not consider the capital costs of installing pipelines for nitrous oxide delivery and the ongoing manpower requirement to maintain the nitrous oxide manifold system. Furthermore, as highlighted in a very recent report,11 technical errors can result in inadvertent hypoxemia that may be fatal or permanently disabling. Also, the ENIGMA trial identified nitrous oxide as a risk factor for serious wound infection and respiratory complication in patients undergoing major noncardiac surgery. These adverse events pose a significant economic burden to any healthcare system.
Dr. Tornero-Campello expresses a number of opinions in response to the editorial by Dr. Hopf that accompanied our article, but we wish to concentrate on his comments related to our study. We strongly disagree with his comment in reference to volatile anesthetics and PONV, viz.  “so you would not get any benefit substituting halogenated anesthetics for nitrous oxide.” Inhaled volatile anesthetics are more emetogenic than propofol, but omission of nitrous oxide independently reduces the risk of PONV.3,4 Our study supports a strong effect across a range of patient and surgical types with respect to severe PONV, the definition of which was based on an extended period of symptoms or failure of therapy. This effect was evident despite the majority of patients in both arms of our study receiving volatile anesthetic and, in the nitrous oxide–free arm, at higher concentration (as might be anticipated). In addition, Dr. Tornero-Campello should not be mystified that other investigators found that the results of a subsequent meta-analysis on the effect of inspired oxygen concentration on PONV contradicted the previous results of a trial they had conducted. There is no paradox in this at all, and the investigators are to be congratulated that they retained their scientific curiosity and the motivation to question the validity of their previous findings based on one small trial.
We agree that the effects of different inspired oxygen concentrations on perioperative outcome have not been adequately investigated. We made it clear in the Discussion that no independent effect of oxygen concentration on outcomes was found in an exploratory analysis, but that it is not possible from our trial to determine with confidence whether the benefits of nitrous oxide–free anesthesia derived from omission of nitrous oxide, increased inspired oxygen concentrations, or both.
Dr. Merckx et al.  comment on the difference between explanatory and pragmatic trials and agree with us that the ENIGMA trial was a pragmatic trial given that the two treatment arms were designed to reflect routine clinical practice: nitrous oxide–based and nitrous oxide–free anesthesia. However, they suggest that we strayed from the true objectives of a pragmatic trial in our classic approach to sample size estimation and statistical analysis of the data. According to Merckx et al.  , the sole objective in every pragmatic trial is to make a decision about which is the better of the two treatment arms being tested. A further consequence of the decision-making focus is that no statistical tests or presentations of statistical uncertainty (i.e.  , confidence intervals, P  values) are required.
We do not agree with Drs. Merckx et al.  that decision making is the defining objective of a pragmatic trial; this approach assumes that results will be definitive and uncertainty is irrelevant. Less extreme conceptions of pragmatic trials that are consistent with our approach are given by others,7,12 and our format for reporting of trial results is consistent with current recommendations for presentation, such as the CONSORT statement.13 
Finally, we would like to conclude by emphasizing that, currently, we believe nitrous oxide still has a role in contemporary anesthetic practice, but such use should be selective and take into account the risk profile of the patient and the surgical procedure. Patients with risk factors for PONV and with comorbidities, who undergo major surgery, are more likely to suffer harm from nitrous oxide exposure. Certainly, further large trials are warranted to explore some of the above unresolved issues.
†Alfred Hospital and Monash University, Melbourne, Victoria, Australia.
Myles PS, Leslie K, Chan MTV, Forbes A, Paech MJ, Peyton P, Silbert BS, Pascoe E, ENIGMA Trial Group: Avoidance of nitrous oxide for patients undergoing major surgery: A randomized controlled trial. Anesthesiology 2007; 107:221–31Myles, PS Leslie, K Chan, MTV Forbes, A Paech, MJ Peyton, P Silbert, BS Pascoe, E ENIGMA Trial Group,
Badner N, Beattie W, Freeman D, Spence J: Nitrous oxide-induced increased homocysteine concentrations are associated with increased postoperative myocardial ischemia in patients undergoing carotid endarterectomy. Anesth Analg 2000; 91:1073–9Badner, N Beattie, W Freeman, D Spence, J
Divatia JV, Vaidya JS, Badwe RA, Hawaldar RW: Omission of nitrous oxide during anesthesia reduces the incidence of postoperative nausea and vomiting: A meta-analysis. Anesthesiology 1996; 85:1055–62Divatia, JV Vaidya, JS Badwe, RA Hawaldar, RW
Tramer M, Moore A, McQuay H: Meta-analytic comparison of prophylactic antiemetic efficacy for postoperative nausea and vomiting: Propofol anaesthesia versus  omitting nitrous oxide versus  total i.v. anaesthesia with propofol. Br J Anaesth 1997; 78:256–9Tramer, M Moore, A McQuay, H
Yusuf S, Collins R, Peto R: Why do we need some large, simple randomized trials? Stat Med 1984; 3:409–20Yusuf, S Collins, R Peto, R
Myles PS: Why we need large trials in anaesthesia and analgesia, An Evidence Based Resource in Anaesthesia and Analgesia, 2nd edition. By Tramer MR. London, BMJ Publishing Group, 2003, pp 12–21
McMahon AD: Study control, violators, inclusion criteria and defining explanatory and pragmatic trials. Stat Med 2002; 21:1365–76McMahon, AD
Tunis SR, Stryer DB, Clancy CM: Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003; 290:1624–32Tunis, SR Stryer, DB Clancy, CM
Joyce C, Baker A, Kennedy R: Gas uptake from an unventilated area of lung: Computer model of absorption atelectasis. J Appl Physiol 1993; 74:1107–16Joyce, C Baker, A Kennedy, R
Tang J, Chen L, White PF, Wender RH, Naruse R, Kariger R, Sloninsky A: Use of propofol for office-based anesthesia: Effect of nitrous oxide on recovery profile. J Clin Anesth 1999; 11:226–30Tang, J Chen, L White, PF Wender, RH Naruse, R Kariger, R Sloninsky, A
Herff H, Paal P, von Goedecke A, Lindner KH, Keller C, Wenzel V: Fatal errors in nitrous oxide delivery. Anaesthesia 2007; 62:1202–6Herff, H Paal, P von Goedecke, A Lindner, KH Keller, C Wenzel, V
Hotopf M, Churchill R, Lewis G: Pragmatic randomised controlled trials in psychiatry. Br J Psychiatry 1999; 175:217–23Hotopf, M Churchill, R Lewis, G
Moher D, Schulz KF, Altman DG: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001; 357:1191–4Moher, D Schulz, KF Altman, DG