Observational Methods for COVID-19 Vaccine Effectiveness Research: an Empirical Evaluation and Target Trial Emulation

Background: There are scarce data on best practices to control for confounding in observational studies assessing vaccine effectiveness to prevent COVID-19. We compared the performance of three well-established methods [overlap weighting, inverse probability treatment weighting and propensity score (PS) matching] to minimize confounding when comparing vaccinated and unvaccinated people. Subsequently, we conducted a target trial emulation to study the ability of these methods to replicate COVID-19 vaccine trials.

Methods: We included all individuals aged ≥75 from primary care records from the UK [Clinical Practice Research Datalink (CPRD) AURUM], who were not infected with or vaccinated against SARS-CoV-2 as of 4 January 2021. Vaccination status was then defined based on first COVID-19 vaccine dose exposure between 4 January 2021 and 28 January 2021. Lasso regression was used to calculate PS. Location, age, prior observation time, regional vaccination rates, testing effort and COVID-19 incidence rates at index date were forced into the PS. Following PS weighting and matching, the three methods were compared for remaining covariate imbalance and residual confounding. Last, a target trial emulation comparing COVID-19 at 3 and 12 weeks after first vaccine dose vs unvaccinated was conducted.

Results: Vaccinated and unvaccinated cohorts comprised 583 813 and 332 315 individuals for weighting, respectively, and 459 000 individuals in the matched cohorts. Overlap weighting performed best in terms of minimizing confounding and systematic error. Overlap weighting successfully replicated estimates from clinical trials for vaccine effectiveness for ChAdOx1 (57%) and BNT162b2 (75%) at 12 weeks.

Conclusion: Overlap weighting performed best in our setting. Our results based on overlap weighting replicate previous pivotal trials for the two first COVID-19 vaccines approved in Europe.