Propensity Score Methods Gave Similar Results to Traditional Regression Modeling in Observational Studies: a Systematic Review

Objective: To determine whether adjusting for confounder bias in observational studies using propensity scores gives different results than using traditional regression modeling.

Methods: Medline and Embase were used to identify studies that described at least one association between an exposure and an outcome using both traditional regression and propensity score methods to control for confounding. From 43 studies, 78 exposure-outcome associations were found. Measures of the quality of propensity score implementation were determined. The statistical significance of each association using both analytical methods was compared. The odds or hazard ratios derived using both methods were compared quantitatively.

Results: Statistical significance differed between regression and propensity score methods for only 8 of the associations (10%), kappa = 0.79 (95% CI = 0.65-0.92). In all cases, the regression method gave a statistically significant association not observed with the propensity score method. The odds or hazard ratio derived using propensity scores was, on average, 6.4% closer to unity than that derived using traditional regression.

Conclusions: Observational studies had similar results whether using traditional regression or propensity scores to adjust for confounding. Propensity scores gave slightly weaker associations; however, many of the reviewed studies did not implement propensity scores well.