Free
Correspondence  |   August 2012
Considerations for Evaluating the Accuracy of Hemoglobin Monitoring
Author Notes
  • Masimo Corporation, Irvine, California.
Article Information
Correspondence
Correspondence   |   August 2012
Considerations for Evaluating the Accuracy of Hemoglobin Monitoring
Anesthesiology 8 2012, Vol.117, 429-430. doi:10.1097/ALN.0b013e31825d95d6
Anesthesiology 8 2012, Vol.117, 429-430. doi:10.1097/ALN.0b013e31825d95d6
To the Editor: 
Masimo manufactures the Radical-7®, a multi-wavelength Pulse CO-Oximeter that continuously measures noninvasive hemoglobin concentration (SpHb®). This technology was the subject of a study by Applegate et al.  , evaluating the accuracy of revision “E” SpHb sensors and software during abdominal and pelvic surgery.1 Masimo appreciates the work of Applegate et al.  , and we are grateful for the opportunity to comment.
Applegate et al.  reported wider SpHb variation from laboratory hemoglobin than some other investigators have seen, and also reported that in some cases, SpHb did not trend in a consistent direction with the laboratory device they used in the study. The authors stated that the laboratory device, an operating room CO-Oximeter, has documented accuracy of ±0 to ±0.2 g/dl based on quality checks performed during the study. These quality checks do not actually assess accuracy but rather device precision (variation) in measurement based on running multiple reference samples. Quality analyses can be misleading because they do not use consecutive clinical blood samples run on the same or multiple laboratory devices. Significant variation in laboratory measurement is introduced by the blood sampling, storage, and mixing technique. For example, without careful attention, withdrawing blood through an arterial or venous line can allow fluid in the line to mix with the blood. Likewise, insufficient or inconsistent mixing allows blood to coagulate and renders hemoglobin measurements inaccurate.
The true accuracy of any laboratory hemoglobin device can only be assessed by comparing it with the international standard for hemoglobin, the hemiglobincyanide method,2 as described by the International Council for Standardization in Hematology and required by the Food and Drug Administration for laboratory device submissions. Because the hemiglobincyanide method is challenging to perform in clinical settings because of complexity and time requirements, the hematology analyzer (e.g.  , Beckman Coulter or Sysmex) is often used as the best available clinical standard.3 Bland and Altman pointed out that both reference devices and test devices produce and contain inherent errors.4 Therefore, a complete picture of SpHb accuracy must be relative, with SpHb and other laboratory devices used clinically today at the point of care, such as operating room CO-Oximeters and portable devices such as i-Stat (Abbott Laboratories, Abbott Park, IL) and Hemocue (HemoCue, Inc., Cypress, CA), compared with the international hemoglobin reference standard, cynanmethemoglobin, or at least to the best-known clinical standard, a hematology analyzer.5 In such studies, it is critical that only one laboratory device of each type be used, as variation exists even within the same device model of different serial numbers – shown to be as high as 0.9 g/dl SD.6 
Reporting the bias and SD of invasive but commonly available laboratory devices along with SpHb in the same subjects provides an objective evaluation of SpHb accuracy, provided proper and consistent blood sampling, storage, and mixing techniques are followed, as well as running the reference sample on a single, appropriate laboratory device, as previously described. Frasca et al.  used a study design like this to evaluate the accuracy of revision E SpHb sensors in 471 comparisons made in the intensive care unit. SpHb, a satellite laboratory CO-Oximeter (RapidPoint 405; Siemens Healthcare Diagnostics Inc., Tarrytown, NY), and a point-of-care device (Hemocue 301) were compared with reference hemoglobin from the central laboratory hematology analyzer (Sysmex XT2000i; Sysmex, Kobe, Japan).7 The bias ± precision of SpHb was 0.0 ± 1.0 g/dl, the CO-Oximeter was 0.9 ± 0.6 g/dl, and the point-of-care device was 0.3 ± 1.3 g/dl. In the same study, changes in SpHb compared with changes in the reference hemoglobin showed the same correlation as the laboratory CO-Oximeter and better correlation than the point-of-care device.
In addition to laboratory device and blood sampling, storage, and mixing, other factors can affect the accuracy of SpHb technology, including initial sensor placement, monitoring of the sensor placement during use for potential misalignment, and use of light shielding. We would also like to point out that SpHb was configured to a long averaging time of approximately 3 min in the study by Applegate et al.  During periods of rapidly changing hemoglobin concentration, the blood sample representing blood over several seconds was compared with SpHb values averaged during approximately 3 min. When hemoglobin concentration is dropping rapidly, this can lead to an overestimation by SpHb. When hemoglobin concentration is rising rapidly, this can lead to an underestimation by SpHb. Lastly, the investigators chose to record SpHb values manually, rather than using an automated data collection method, which can introduce error into the study results, especially if hemoglobin is changing rapidly. Masimo makes available data collection software that allows for time stamping of blood draws and other events during SpHb data collection.
In the analysis technique used by Applegate et al.  in their figure 3 scatterplot, the change in SpHb is plotted versus  the change in laboratory hemoglobin. Because of variability in laboratory hemoglobin from the aforementioned factors, small changes in hemoglobin, such as those under 2.0 g/dl, should not be compared with SpHb changes. Other investigators have performed similar analyses in which reference data points with small magnitude changes were removed.8 Critchley et al.  have proposed a “polar plot” technique that may provide the optimal method to evaluate trending ability by taking into account both bias and the magnitude of the changes.9 
Masimo is proud of the innovations we have brought to monitoring, as well as our ability to rapidly improve technologies. The absolute accuracy reported by Applegate et al.  is similar to that reported by Miller et al.  in complex spine surgery, but significantly different than at least two other published studies in surgery evaluating SpHb revision E technology.10 Berkow et al.  also evaluated revision E SpHb sensors in complex spine surgery and reported a 1.0 g/dl SD and clinically acceptable trend accuracy.8 Lamhaut et al.  evaluated revision E in major urologic surgery and showed a similar 1.1 g/dl SD, whereas a point of care device showed a 0.7 g/dl SD.11 
We are confident that SpHb will reduce inappropriate blood transfusions during periods of visible blood loss but with stable hemoglobin status, and will enable earlier detection of occult bleeding. We believe these evaluations have greater clinical relevance than point-to-point accuracy comparisons. A randomized controlled trial has already been presented in abstract form that showed decrease in blood transfusion frequency (from 4.5 to 0.6%) in orthopedic surgery patients monitored with SpHb compared with a group managed by standard care, with no negative impact on patient safety.12 We expect this to be the first of many studies showing SpHb's impact on patient care.
References
Applegate RL 2nd, Barr SJ, Collier CE, Rook JL, Mangus DB, Allard MW: Evaluation of pulse cooximetry in patients undergoing abdominal or pelvic surgery. ANESTHESIOLOGY 2011; 116:65–72
Recommendations for reference method for haemoglobinometry in human blood (ICSH standard 1986) and specifications for international haemiglobincyanide reference preparation (3rd edition). International Committee for Standardization in Haematology; Expert Panel on Haemoglobinometry. Clin Lab Haematol 1987; 9:73–9
Takubo T, Tatsumi N, Satoh N, Matsuno K, Fujimoto K, Soga M, Yamagami Y, Akiba S, Sudoh T, Miyazaki M: Evaluation of hematological values obtained with reference automated hematology analyzers of six manufacturers. Southeast Asian J Trop Med Public Health 2002; 33 Suppl 2:62–7
Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307–10
Zwart A, van Assendelft OW, Bull BS, England JM, Lewis SM, Zijlstra WG: Recommendations for reference method for haemoglobinometry in human blood (ICSH standard 1995) and specifications for international haemiglobinocyanide standard (4th edition). J Clin Pathol 1996; 49:271–4
Gehring H, Duembgen L, Peterlein M, Hagelberg S, Dibbelt L: Hemoximetry as the “gold standard”? Error assessment based on differences among identical blood gas analyzer devices of five manufacturers. Anesth Analg 2007; 105:S24–30
Frasca D, Dahyot-Fizelier C, Catherine K, Levrat Q, Debaene B, Mimoz O: Accuracy of a continuous noninvasive hemoglobin monitor in intensive care unit patients. Crit Care Med 2011; 39:2277–82
Berkow L, Rotolo S, Mirski E: Continuous noninvasive hemoglobin monitoring during complex spine surgery. Anesth Analg 2011; 113:1396–402
Critchley LA, Lee A, Ho AM: A critical review of the ability of continuous cardiac output monitors to measure trends in cardiac output. Anesth Analg 2010; 111:1180–92
Miller RD, Ward TA, Shiboski SC, Cohen NH: A comparison of three methods of hemoglobin monitoring in patients undergoing spine surgery. Anesth Analg 2011; 112:858–63
Lamhaut L, Apriotesei R, Combes X, Lejay M, Carli P, Vivien B: Comparison of the accuracy of noninvasive hemoglobin monitoring by spectrophotometry (SpHb) and HemoCue® with automated laboratory hemoglobin measurement. ANESTHESIOLOGY 2011; 115:548–54
Ehrenfeld JM, Henneman JP: Impact of continuous and noninvasive hemoglobin monitoring on intraoperative blood transfusions. Paper presented at: Annual Meeting of the American Society of Anesthesiologists; October 18, 2010; San Diego, CA