Correspondence  |   September 2018
The Curse of Dimensionality
Author Notes
  • University of Vermont Larner College of Medicine, Burlington, Vermont (N.P.P.).
  • (Accepted for publication June 8, 2018.)
    (Accepted for publication June 8, 2018.)×
Article Information
Correspondence   |   September 2018
The Curse of Dimensionality
Anesthesiology 9 2018, Vol.129, 614-615. doi:10.1097/ALN.0000000000002350
Anesthesiology 9 2018, Vol.129, 614-615. doi:10.1097/ALN.0000000000002350
In a recent article, Kheterpal et al.1  analyzed the impact of a real-time intraoperative decision support system. Borrowing tactics from the aviation industry, the authors hypothesized that “decision support systems, which integrate across disparate data sources, devices, and contexts, to highlight and recommend specific interventions” might lead to better postoperative outcomes. For now, the authors showed that these systems did improve process measures, but the clinical outcomes were lacking. These results are not surprising.
In the field of data science, researchers understand that the “curse of dimensionality” is lurking behind every hypothesis.2  Here, the introduction of additional dimensions waters down the “relative contrast” of each data point and they become clustered together. One is no longer looking at data points on an x,y plane and one cannot differentiate the “distance,” or significance, of each point.3  As a result, one may observe a statistical significance when analyzing the data in its entirety, but in reality, it may only be in a subset of data points. Further, the aggregation of large amounts of data may inadvertently create a collection of irrelevant, correlated, or redundant data, interfering with any subsequent analyses.3  For example, heart rate and blood pressure are commonly inversely related and correlated to a different degree depending on the scenario. The variation in correlation forces the researcher to account for these differences when analyzing the data. Finally, when patterns are uncovered with insufficient data, the model may have statistical significance, but the overall utility/effect size may not justify the means.4