Bench-to-bedside Review: the Importance of the Precision of the Reference Technique in Method Comparison Studies--with Specific Reference to the Measurement of Cardiac Output

Overview

Journal Crit Care

Specialty Critical Care

Date 2009 Feb 3

PMID 19183431

Citations 120

Authors

Maurizio Cecconi

Andrew Rhodes

Jan Poloniecki

Giorgio Della Rocca

R Michael Grounds

Affiliations

Soon will be listed here.

Abstract

Bland-Altman analysis is used for assessing agreement between two measurements of the same clinical variable. In the field of cardiac output monitoring, its results, in terms of bias and limits of agreement, are often difficult to interpret, leading clinicians to use a cutoff of 30% in the percentage error in order to decide whether a new technique may be considered a good alternative. This percentage error of +/- 30% arises from the assumption that the commonly used reference technique, intermittent thermodilution, has a precision of +/- 20% or less. The combination of two precisions of +/- 20% equates to a total error of +/- 28.3%, which is commonly rounded up to +/- 30%. Thus, finding a percentage error of less than +/- 30% should equate to the new tested technique having an error similar to the reference, which therefore should be acceptable. In a worked example in this paper, we discuss the limitations of this approach, in particular in regard to the situation in which the reference technique may be either more or less precise than would normally be expected. This can lead to inappropriate conclusions being drawn from data acquired in validation studies of new monitoring technologies. We conclude that it is not acceptable to present comparison studies quoting percentage error as an acceptability criteria without reporting the precision of the reference technique.

Citing Articles

Comparison of noninvasive electrical cardiometry and transpulmonary thermodilution for cardiac output measurement in critically ill patients: a prospective observational study.

Song W, Guo J, Cao D, Jiang J, Yang T, Ma X BMC Anesthesiol. 2025; 25(1):123.

PMID: 40082773 PMC: 11905673. DOI: 10.1186/s12871-025-03005-1.

Validation of the capnodynamic method to calculate mixed venous oxygen saturation in postoperative cardiac patients.

Wallin M, Hallback M, Iftikhar H, Keleher E, Aneman A Intensive Care Med Exp. 2025; 13(1):32.

PMID: 40053202 PMC: 11889286. DOI: 10.1186/s40635-025-00741-z.

DiCART device to measure capillary refill time: a validation study in patients with acute circulatory failure.

Descamps A, Jacquet-Lagreze M, Aussal T, Fellahi J, Ruste M J Clin Monit Comput. 2025; .

PMID: 40011397 DOI: 10.1007/s10877-025-01271-5.

Capnodynamic determination of end-expiratory lung volume in a porcine model of hypoxic pulmonary vasoconstriction.

Tornwall A, Wallin M, Hallback M, Lonnqvist P, Karlsson J J Clin Monit Comput. 2024; .

PMID: 39665870 DOI: 10.1007/s10877-024-01251-1.

Assessing the repeatability, reliability, and precision of right ventricular outflow tract and mid-pulmonary artery diameters, velocity time integrals, and agreement between site-specific stroke volumes.

Mohseni-Badalabadi R, Hosseininejad L, Hali R, Fallah F, Hosseinsabet A BMC Cardiovasc Disord. 2024; 24(1):703.

PMID: 39639197 PMC: 11622495. DOI: 10.1186/s12872-024-04384-y.

References

Bland J, Altman D . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1(8476):307-10. View

Bland J, Altman D . Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995; 346(8982):1085-7. DOI: 10.1016/s0140-6736(95)91748-9. View

Critchley L, Critchley J . A meta-analysis of studies using bias and precision statistics to compare cardiac output measurement techniques. J Clin Monit Comput. 2003; 15(2):85-91. DOI: 10.1023/a:1009982611386. View

Bland J, Altman D . Measuring agreement in method comparison studies. Stat Methods Med Res. 1999; 8(2):135-60. DOI: 10.1177/096228029900800204. View

Della Rocca G, Costa M, Pompei L, Coccia C, Pietropaoli P . Continuous and intermittent cardiac output measurement: pulmonary artery catheter versus aortic transpulmonary technique. Br J Anaesth. 2002; 88(3):350-6. DOI: 10.1093/bja/88.3.350. View

Bein B, Worthmann F, Tonner P, Paris A, Steinfath M, Hedderich J . Comparison of esophageal Doppler, pulse contour analysis, and real-time pulmonary artery thermodilution for the continuous measurement of cardiac output. J Cardiothorac Vasc Anesth. 2004; 18(2):185-9. DOI: 10.1053/j.jvca.2004.01.025. View

Barin E, Haryadi D, Schookin S, Westenskow D, ZUBENKO V, Beliaev K . Evaluation of a thoracic bioimpedance cardiac output monitor during cardiac catheterization. Crit Care Med. 2001; 28(3):698-702. DOI: 10.1097/00003246-200003000-00016. View

Lichtenthal P, Gordan D . Testing the safety of Baxter continuous cardiac output monitoring system. J Clin Monit. 1996; 12(3):243-9. DOI: 10.1007/BF00857646. View

Jakobsen C, Melsen N, Andresen E . Continuous cardiac output measurements in the perioperative period. Acta Anaesthesiol Scand. 1995; 39(4):485-8. DOI: 10.1111/j.1399-6576.1995.tb04104.x. View

10.

Hamilton T, Huber L, Jessen M . PulseCO: a less-invasive method to monitor cardiac output from arterial pressure after cardiac surgery. Ann Thorac Surg. 2002; 74(4):S1408-12. DOI: 10.1016/s0003-4975(02)04059-6. View

11.

Munro H, Wood C, Taylor B, Smith G . Continuous invasive cardiac output monitoring--the Baxter/Edwards Critical-Care Swan Ganz IntelliCath and Viligance system. Clin Intensive Care. 1993; 5(2):52-5. View

12.

Boyd O, Grounds R, Bennett E . A randomized clinical trial of the effect of deliberate perioperative increase of oxygen delivery on mortality in high-risk surgical patients. JAMA. 1993; 270(22):2699-707. View

13.

Neviere R, Mathieu D, Riou Y, Chagnon J, WATTEL F . Non-invasive measurement of cardiac output in patients with acute lung injury using the carbon dioxide rebreathing method. Clin Intensive Care. 1993; 5(4):172-5. View

14.

Roeck M, Jakob S, Boehlen T, Brander L, Knuesel R, Takala J . Change in stroke volume in response to fluid challenge: assessment using esophageal Doppler. Intensive Care Med. 2003; 29(10):1729-35. DOI: 10.1007/s00134-003-1720-4. View

15.

Pearse R, Dawson D, Fawcett J, Rhodes A, Grounds R, Bennett E . Early goal-directed therapy after major surgery reduces complications and duration of hospital stay. A randomised, controlled trial [ISRCTN38797445]. Crit Care. 2005; 9(6):R687-93. PMC: 1414018. DOI: 10.1186/cc3887. View

16.

Button D, Weibel L, Reuthebuch O, Genoni M, Zollinger A, Hofer C . Clinical evaluation of the FloTrac/Vigileo system and two established continuous cardiac output monitoring devices in patients undergoing cardiac surgery. Br J Anaesth. 2007; 99(3):329-36. DOI: 10.1093/bja/aem188. View

17.

Goedje O, Hoeke K, Lichtwarck-Aschoff M, Faltchauser A, Lamm P, Reichart B . Continuous cardiac output by femoral arterial thermodilution calibrated pulse contour analysis: comparison with pulmonary arterial thermodilution. Crit Care Med. 1999; 27(11):2407-12. DOI: 10.1097/00003246-199911000-00014. View

18.

Pittman J, Bar-Yosef S, SumPing J, Sherwood M, Mark J . Continuous cardiac output monitoring with pulse contour analysis: a comparison with lithium indicator dilution cardiac output measurement. Crit Care Med. 2005; 33(9):2015-21. DOI: 10.1097/01.ccm.0000179021.36805.1f. View

19.

Mayer J, Boldt J, Schollhorn T, Rohm K, Mengistu A, Suttner S . Semi-invasive monitoring of cardiac output by a new device using arterial pressure waveform analysis: a comparison with intermittent pulmonary artery thermodilution in patients undergoing cardiac surgery. Br J Anaesth. 2007; 98(2):176-82. DOI: 10.1093/bja/ael341. View

20.

Sander M, Spies C, Grubitzsch H, Foer A, Muller M, von Heymann C . Comparison of uncalibrated arterial waveform analysis in cardiac surgery patients with thermodilution cardiac output measurements. Crit Care. 2006; 10(6):R164. PMC: 1794471. DOI: 10.1186/cc5103. View