Target Validity: Bringing Treatment of External Validity in Line with Internal Validity

Overview

Journal Curr Epidemiol Rep

Specialty Public Health

Date 2021 Feb 15

PMID 33585162

Citations 9

Authors

Catherine R Lesko

Benjamin Ackerman

Michael Webster-Clark

Jessie K Edwards

Affiliations

Soon will be listed here.

Abstract

Purpose Of Review: "Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity.

Recent Findings: External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available.

Summary: The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.

Citing Articles

Four targets: an enhanced framework for guiding causal inference from observational data.

Lu H, Li F, Lesko C, Fink D, Rudolph K, Harhay M Int J Epidemiol. 2025; 54(1).

PMID: 39868475 PMC: 11769716. DOI: 10.1093/ije/dyaf003.

Adverse Events During a 12-month Multi-Site and Dose-Response Aerobic Exercise Intervention.

Vidoni E, Grove G, Szabo-Reed A, Key M, Huang H, Burns J medRxiv. 2024; .

PMID: 39314981 PMC: 11419234. DOI: 10.1101/2024.09.10.24313352.

How Generalizable Are Findings from a Community-Based Prospective Cohort Study? Extending Estimates from the Adult Changes in Thought Study to Its Source Population.

Gibbons L, Mobley T, Mayeda E, Lee C, Gatto N, LaCroix A J Alzheimers Dis. 2024; 100(1):163-174.

PMID: 38848188 PMC: 11423796. DOI: 10.3233/JAD-240247.

Standardizing to specific target populations in distributed networks and multisite pharmacoepidemiologic studies.

Webster-Clark M, Filion K, Platt R Am J Epidemiol. 2024; 193(7):1031-1039.

PMID: 38412261 PMC: 11520739. DOI: 10.1093/aje/kwae015.

Development and evaluation of objective trial performance metrics for multisite clinical studies: Experience from the AlcHep Network.

Dasarathy S, Tu W, Bellar A, Welch N, Kettler C, Tang Q Contemp Clin Trials. 2024; 138:107437.

PMID: 38215876 PMC: 11866342. DOI: 10.1016/j.cct.2024.107437.

References

Rothman K, Hatch E, Gallacher J . Representativeness is not helpful in studying heterogeneity of effects across subgroups. Int J Epidemiol. 2014; 43(2):633-4. DOI: 10.1093/ije/dyt265. View

Tunis S, Stryer D, Clancy C . Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003; 290(12):1624-32. DOI: 10.1001/jama.290.12.1624. View

Greenland S, Pearl J, Robins J . Causal diagrams for epidemiologic research. Epidemiology. 1999; 10(1):37-48. View

Wang C, Mollan K, Hudgens M, Tucker J, Zheng H, Tang W . Generalisability of an online randomised controlled trial: an empirical analysis. J Epidemiol Community Health. 2017; 72(2):173-178. PMC: 5880638. DOI: 10.1136/jech-2017-209976. View

Dekkers O, von Elm E, Algra A, Romijn J, Vandenbroucke J . How to assess the external validity of therapeutic trials: a conceptual approach. Int J Epidemiol. 2009; 39(1):89-94. DOI: 10.1093/ije/dyp174. View

Concato J, Shah N, Horwitz R . Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000; 342(25):1887-92. PMC: 1557642. DOI: 10.1056/NEJM200006223422507. View

Dahabreh I, Hernan M . Extending inferences from a randomized trial to a target population. Eur J Epidemiol. 2019; 34(8):719-722. DOI: 10.1007/s10654-019-00533-2. View

Pressler T, Kaizar E . The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias. Stat Med. 2013; 32(20):3552-68. PMC: 3967586. DOI: 10.1002/sim.5802. View

Hoertel N, Le Strat Y, Blanco C, Lavaud P, Dubertret C . Generalizability of clinical trial results for generalized anxiety disorder to community samples. Depress Anxiety. 2012; 29(7):614-20. DOI: 10.1002/da.21937. View

10.

Rothwell P . Commentary: External validity of results of randomized trials: disentangling a complex concept. Int J Epidemiol. 2009; 39(1):94-6. DOI: 10.1093/ije/dyp305. View

11.

Geller S, Koch A, Roesch P, Filut A, Hallgren E, Carnes M . The More Things Change, the More They Stay the Same: A Study to Evaluate Compliance With Inclusion and Assessment of Women and Minorities in Randomized Controlled Trials. Acad Med. 2017; 93(4):630-635. PMC: 5908758. DOI: 10.1097/ACM.0000000000002027. View

12.

Nguyen T, Ackerman B, Schmid I, Cole S, Stuart E . Sensitivity analyses for effect modifiers not observed in the target population when generalizing treatment effects from a randomized controlled trial: Assumptions, models, effect scales, data scenarios, and implementation details. PLoS One. 2018; 13(12):e0208795. PMC: 6289424. DOI: 10.1371/journal.pone.0208795. View

13.

Stuart E, Bradshaw C, Leaf P . Assessing the generalizability of randomized trial results to target populations. Prev Sci. 2014; 16(3):475-85. PMC: 4359056. DOI: 10.1007/s11121-014-0513-z. View

14.

Steg P, Lopez-Sendon J, Lopez de Sa E, Goodman S, Gore J, Anderson Jr F . External validity of clinical trials in acute myocardial infarction. Arch Intern Med. 2007; 167(1):68-73. DOI: 10.1001/archinte.167.1.68. View

15.

Buchanan A, Hudgens M, Cole S, Mollan K, Sax P, Daar E . Generalizing Evidence from Randomized Trials using Inverse Probability of Sampling Weights. J R Stat Soc Ser A Stat Soc. 2018; 181(4):1193-1209. PMC: 6289264. DOI: 10.1111/rssa.12357. View

16.

Insel T . Beyond efficacy: the STAR*D trial. Am J Psychiatry. 2006; 163(1):5-7. PMC: 1586101. DOI: 10.1176/appi.ajp.163.1.5. View

17.

Susukida R, Crum R, Stuart E, Ebnesajjad C, Mojtabai R . Assessing sample representativeness in randomized controlled trials: application to the National Institute of Drug Abuse Clinical Trials Network. Addiction. 2016; 111(7):1226-34. PMC: 4899104. DOI: 10.1111/add.13327. View

18.

Westreich D, Edwards J, Lesko C, Cole S, Stuart E . Target Validity and the Hierarchy of Study Designs. Am J Epidemiol. 2018; 188(2):438-443. PMC: 6357801. DOI: 10.1093/aje/kwy228. View

19.

Rothwell P . External validity of randomised controlled trials: "to whom do the results of this trial apply?". Lancet. 2005; 365(9453):82-93. DOI: 10.1016/S0140-6736(04)17670-8. View

20.

Marcus S . Assessing non-consent bias with parallel randomized and nonrandomized clinical trials. J Clin Epidemiol. 1997; 50(7):823-8. DOI: 10.1016/s0895-4356(97)00068-1. View