Comparison Between Treatment Effects in a Randomised Controlled Trial and an Observational Study Using Propensity Scores in Primary Care

Overview

Journal Br J Gen Pract

Specialty Public Health

Date 2017 Aug 2

PMID 28760739

Citations 8

Authors

Beth L Stuart

Louise En Grebel

Christopher C Butler

Kerenza Hood

Theo J M Verheij

Paul Little

Affiliations

Soon will be listed here.

Abstract

Background: Although randomised controlled trials (RCTs) are considered 'gold standard' evidence, they are not always feasible or appropriate, and may represent a select population. Observational studies provide a useful alternative to enhance applicability, but results can be biased due to confounding.

Aim: To explore the utility of propensity scores for causal inference in an observational study.

Design And Setting: Comparison of the effect of amoxicillin on key outcomes in an international RCT and observational study of lower respiratory tract infections.

Method: Propensity scores were calculated and applied as probability weights in the analyses. The adjusted results were compared with the effects reported in the RCT.

Results: Groups were well balanced in the RCT but significantly imbalanced in the observational study, with evidence of confounding by indication: patients receiving antibiotics tended to be older and more unwell at baseline consultation. In the trial duration of symptoms (hazard ratio 1.06, 95% CI = 0.96 to 1.18) and symptom severity (-0.07, 95% CI = -0.15 to 0.007) did not differ between groups. Weighting by propensity score in the observational study resulted in very similar estimates of effect: duration of symptoms (hazard ratio 1.06, 95% CI = 0.80 to 1.40) and difference for symptom severity (-0.07, 95% CI = -0.34 to 0.20).

Conclusion: The observational study, after conditioning on propensity score, echoed the trial results. Provided that detailed information is available on potential sources of confounding, effects of interventions can probably be assessed reasonably well in observational datasets, allowing them to be more directly compared with the results of RCTs.

Citing Articles

Cardiac Biomarkers Predict Major Adverse Cardiac Events (MACE) in Incident Haemodialysis Patients: Results from a Global Federated Database.

Davies E, Buckley B, Austin P, Lip G, Rao A, McDowell G Biomedicines. 2025; 13(2).

PMID: 40002781 PMC: 11853129. DOI: 10.3390/biomedicines13020367.

Routine cardiac biomarkers for the prediction of incident major adverse cardiac events in patients with glomerulonephritis: a real-world analysis using a global federated database.

Davies E, Buckley B, Austin P, Lip G, Oni L, McDowell G BMC Nephrol. 2024; 25(1):233.

PMID: 39039475 PMC: 11265111. DOI: 10.1186/s12882-024-03667-y.

Postoperative recreational sports and subjective shoulder function: a comprehensive analysis following intramedullary stabilization of displaced midshaft clavicular fractures.

Langenhan R, Probst A, Bushuven S, Bushuven S, Trifunovic-Koenig M Arch Orthop Trauma Surg. 2024; 144(6):2619-2629.

PMID: 38703216 DOI: 10.1007/s00402-024-05319-0.

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study.

Toews I, Anglemyer A, Nyirenda J, Alsaid D, Balduzzi S, Grummich K Cochrane Database Syst Rev. 2024; 1:MR000034.

PMID: 38174786 PMC: 10765475. DOI: 10.1002/14651858.MR000034.pub3.

Comparison of median sternotomy closure-related complication rates using orthopedic wire or suture in dogs: A multi-institutional observational treatment effect analysis.

Pilot M, Lutchman A, Hennet J, Anderson D, Robinson W, Rossanese M Vet Surg. 2022; 51(6):990-1001.

PMID: 35765719 PMC: 9543654. DOI: 10.1111/vsu.13846.

References

Harder V, Stuart E, Anthony J . Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychol Methods. 2010; 15(3):234-49. PMC: 2936698. DOI: 10.1037/a0019623. View

DAgostino Jr R . Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med. 1998; 17(19):2265-81. DOI: 10.1002/(sici)1097-0258(19981015)17:19<2265::aid-sim918>3.0.co;2-b. View

Normand S, Landrum M, Guadagnoli E, Ayanian J, Ryan T, Cleary P . Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001; 54(4):387-98. DOI: 10.1016/s0895-4356(00)00321-8. View

Klungel O, Martens E, Psaty B, Grobbee D, Sullivan S, Stricker B . Methods to assess intended effects of drug treatment in observational studies are reviewed. J Clin Epidemiol. 2004; 57(12):1223-31. DOI: 10.1016/j.jclinepi.2004.03.011. View

Grootendorst D, Jager K, Zoccali C, Dekker F . Observational studies are complementary to randomized controlled trials. Nephron Clin Pract. 2009; 114(3):c173-7. DOI: 10.1159/000262299. View

McCandless L, Gustafson P, Austin P . Bayesian propensity score analysis for observational data. Stat Med. 2008; 28(1):94-112. DOI: 10.1002/sim.3460. View

Francis N, Gillespie D, Nuttall J, Hood K, Little P, Verheij T . Antibiotics for acute cough: an international observational study of patient adherence in primary care. Br J Gen Pract. 2012; 62(599):e429-37. PMC: 3361123. DOI: 10.3399/bjgp12X649124. View

Hamoen M, Broekhuizen B, Little P, Melbye H, Coenen S, Goossens H . Medication use in European primary care patients with lower respiratory tract infection: an observational study. Br J Gen Pract. 2014; 64(619):e81-91. PMC: 3905423. DOI: 10.3399/bjgp14X677130. View

Weitzen S, Lapane K, Toledano A, Hume A, Mor V . Principles for modeling propensity scores in medical research: a systematic literature review. Pharmacoepidemiol Drug Saf. 2004; 13(12):841-53. DOI: 10.1002/pds.969. View

10.

Sedgwick P . Randomised controlled trials: balance in baseline characteristics. BMJ. 2014; 349:g5721. DOI: 10.1136/bmj.g5721. View

11.

Shah B, Laupacis A, Hux J, Austin P . Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J Clin Epidemiol. 2005; 58(6):550-9. DOI: 10.1016/j.jclinepi.2004.10.016. View

12.

McMahon A . Approaches to combat with confounding by indication in observational studies of intended drug effects. Pharmacoepidemiol Drug Saf. 2003; 12(7):551-8. DOI: 10.1002/pds.883. View

13.

Hajage D, Tubach F, Steg P, Bhatt D, De Rycke Y . On the use of propensity scores in case of rare exposure. BMC Med Res Methodol. 2016; 16:38. PMC: 4815252. DOI: 10.1186/s12874-016-0135-1. View

14.

Li F, Zaslavsky A, Landrum M . Propensity score weighting with multilevel data. Stat Med. 2013; 32(19):3373-87. PMC: 3710526. DOI: 10.1002/sim.5786. View

15.

Little P, Stuart B, Moore M, Coenen S, Butler C, Godycki-Cwirko M . Amoxicillin for acute lower-respiratory-tract infection in primary care when pneumonia is not suspected: a 12-country, randomised, placebo-controlled trial. Lancet Infect Dis. 2012; 13(2):123-9. DOI: 10.1016/S1473-3099(12)70300-6. View

16.

Austin P . An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011; 46(3):399-424. PMC: 3144483. DOI: 10.1080/00273171.2011.568786. View

17.

Brookhart M, Schneeweiss S, Rothman K, Glynn R, Avorn J, Sturmer T . Variable selection for propensity score models. Am J Epidemiol. 2006; 163(12):1149-56. PMC: 1513192. DOI: 10.1093/aje/kwj149. View

18.

Shadish W, Galindo R, Wong V, Steiner P, Cook T . A randomized experiment comparing random and cutoff-based assignment. Psychol Methods. 2011; 16(2):179-91. DOI: 10.1037/a0023345. View

19.

Braitman L, Rosenbaum P . Rare outcomes, common treatments: analytic strategies using propensity scores. Ann Intern Med. 2002; 137(8):693-5. DOI: 10.7326/0003-4819-137-8-200210150-00015. View

20.

Williamson E, Forbes A . Introduction to propensity scores. Respirology. 2014; 19(5):625-35. DOI: 10.1111/resp.12312. View