Counterintuitive Results from Observational Data: a Case Study and Discussion

Overview

Journal BMJ Open

Specialty General Medicine

Date 2019 May 8

PMID 31061037

Citations 1

Authors

Erik Doty

David J Stone

Ned McCague

Leo Anthony Celi

Affiliations

Soon will be listed here.

Abstract

Objective: To explore the issue of counterintuitive data via analysis of a representative case in which the data obtained was unexpected and inconsistent with current knowledge. We then discuss the issue of counterintuitive data while developing a framework for approaching such findings.

Design: The case study is a retrospective analysis of a cohort of coronary artery bypass graft (CABG) patients. Regression was used to examine the association between perceived pain in the intensive care unit (ICU) and selected outcomes.

Setting: Medical Information Mart for Intensive Care-III, a publicly available, de-identified critical care patient database.

Participants: 844 adult patients from the database who underwent CABG surgery and were extubated within 24 hours after ICU admission.

Outcomes: 30 day mortality, 1 year mortality and hospital length of stay (LOS).

Results: Increased pain levels were found to be significantly associated with reduced mortality at 30 days and 1 year, and shorter hospital LOS. A one-point increase in mean pain level was found to be associated with a reduction in the odds of 30 day and 1 year mortality by a factor of 0.457 (95% CI 0.304 to 0.687, p<0.01) and 0.710 (95% CI 0.571 to 0.881, p<0.01) respectively, and a 0.916 (95% CI -1.159 to -0.673, p<0.01) day decrease in hospital LOS.

Conclusion: The finding of an association between increased pain and improved outcomes was unexpected and clinically counterintuitive. In an increasingly digitised age of medical big data, such results are likely to become more common. The reliability of such counterintuitive results must be carefully examined. We suggest several issues to consider in this analytic process. If the data is determined to be valid, consideration must then be made towards alternative explanations for the counterintuitive results observed. Such results may in fact indicate that current clinical knowledge is incomplete or not have been firmly based on empirical evidence and function to inspire further research into the factors involved.

Citing Articles

Effect of heart rate on hospital mortality in critically ill patients may be modified by age: a retrospective observational study from large database.

Zhou D, Li Z, Shi G, Zhou J Aging Clin Exp Res. 2020; 33(5):1325-1335.

PMID: 32638341 DOI: 10.1007/s40520-020-01644-7.

References

Hine L, Laird N, Hewitt P, CHALMERS T . Meta-analytic evidence against prophylactic use of lidocaine in acute myocardial infarction. Arch Intern Med. 1989; 149(12):2694-8. View

Prasad V, Jena A . Prespecified falsification end points: can they validate true observational associations?. JAMA. 2013; 309(3):241-2. DOI: 10.1001/jama.2012.96867. View

Ljungqvist O, Scott M, Fearon K . Enhanced Recovery After Surgery: A Review. JAMA Surg. 2017; 152(3):292-298. DOI: 10.1001/jamasurg.2016.4952. View

Bykowski M, Sivak W, Garland C, Cladis F, Goldstein J, Losee J . A Multimodal Preemptive Analgesic Protocol for Alveolar Bone Graft Surgery: Decreased Pain, Hospital Stay, and Health Care Costs. Cleft Palate Craniofac J. 2018; 56(4):479-486. DOI: 10.1177/1055665618791943. View

Connors Jr A, Speroff T, Dawson N, Thomas C, Harrell Jr F, Wagner D . The effectiveness of right heart catheterization in the initial care of critically ill patients. SUPPORT Investigators. JAMA. 1996; 276(11):889-97. DOI: 10.1001/jama.276.11.889. View

Werner S, Grose R . Regulation of wound healing by growth factors and cytokines. Physiol Rev. 2003; 83(3):835-70. DOI: 10.1152/physrev.2003.83.3.835. View

Watkins L, Milligan E, Maier S . Glial proinflammatory cytokines mediate exaggerated pain states: implications for clinical pain. Adv Exp Med Biol. 2003; 521:1-21. View

Seitz K, Sevransky J, Martin G, Roback J, Murphy D . Evaluation of RBC Transfusion Practice in Adult ICUs and the Effect of Restrictive Transfusion Protocols on Routine Care. Crit Care Med. 2016; 45(2):271-281. PMC: 5547397. DOI: 10.1097/CCM.0000000000002077. View

Lewis K, Whipple J, Michael K, Quebbeman E . Effect of analgesic treatment on the physiological consequences of acute pain. Am J Hosp Pharm. 1994; 51(12):1539-54. View

10.

Rodgers A, Walker N, Schug S, McKee A, Kehlet H, van Zundert A . Reduction of postoperative mortality and morbidity with epidural or spinal anaesthesia: results from overview of randomised trials. BMJ. 2000; 321(7275):1493. PMC: 27550. DOI: 10.1136/bmj.321.7275.1493. View

11.

Wu C, Hurley R, Anderson G, Herbert R, Rowlingson A, Fleisher L . Effect of postoperative epidural analgesia on morbidity and mortality following surgery in medicare patients. Reg Anesth Pain Med. 2005; 29(6):525-33. DOI: 10.1016/j.rapm.2004.07.002. View

12.

Shroyer A, Coombs L, Peterson E, Eiken M, DeLong E, Chen A . The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg. 2003; 75(6):1856-64; discussion 1864-5. DOI: 10.1016/s0003-4975(03)00179-6. View

13.

Gajic O, Dara S, Mendez J, Adesanya A, Festic E, Caples S . Ventilator-associated lung injury in patients without acute lung injury at the onset of mechanical ventilation. Crit Care Med. 2004; 32(9):1817-24. DOI: 10.1097/01.ccm.0000133019.52531.30. View

14.

Johnson A, Kramer A, Clifford G . A new severity of illness scale using a subset of Acute Physiology And Chronic Health Evaluation data elements shows comparable predictive accuracy. Crit Care Med. 2013; 41(7):1711-8. DOI: 10.1097/CCM.0b013e31828a24fe. View

15.

Wang J, Klein H . Red blood cell transfusion in the treatment and management of anaemia: the search for the elusive transfusion trigger. Vox Sang. 2009; 98(1):2-11. DOI: 10.1111/j.1423-0410.2009.01223.x. View

16.

Johnson A, Pollard T, Shen L, Lehman L, Feng M, Ghassemi M . MIMIC-III, a freely accessible critical care database. Sci Data. 2016; 3:160035. PMC: 4878278. DOI: 10.1038/sdata.2016.35. View

17.

Zhang J, An J . Cytokines, inflammation, and pain. Int Anesthesiol Clin. 2007; 45(2):27-37. PMC: 2785020. DOI: 10.1097/AIA.0b013e318034194e. View

18.

Sutherland J, Liu G, Crump T, Bair M, Karimuddin A . Relationship between preoperative patient-reported outcomes and hospital length of stay: a prospective cohort study of general surgery patients in Vancouver, Canada. J Health Serv Res Policy. 2018; 24(1):29-36. DOI: 10.1177/1355819618791634. View

19.

Hawkes A, Nowak M, Bidstrup B, Speare R . Outcomes of coronary artery bypass graft surgery. Vasc Health Risk Manag. 2007; 2(4):477-84. PMC: 1994021. DOI: 10.2147/vhrm.2006.2.4.477. View

20.

Elixhauser A, Steiner C, Harris D, Coffey R . Comorbidity measures for use with administrative data. Med Care. 1998; 36(1):8-27. DOI: 10.1097/00005650-199801000-00004. View