Causal Conclusions Are Most Sensitive to Unobserved Binary Covariates

There is a rich literature that considers whether an observed relation between treatment and response is due to an unobserved covariate. In order to quantify this unmeasured bias, an assumption is made about the distribution of this unobserved covariate; typically that it is either binary or at least confined to the unit interval. In this paper, this assumption is relaxed in the context of matched pairs with binary treatment and response. One might think that a long-tailed unobserved covariate could do more damage. Remarkably that is not the case: the most harm is done by a binary covariate, so the case commonly considered in the literature is most conservative. This has two practical consequences: (i) it is always safe to assume that an unobserved covariate is binary, if one is content to make a conservative statement; (ii) when another assumption seems more appropriate, say normal covariate, there will be less sensitivity than with a binary covariate. This assumption implies that it is possible that a relation between treatment and response that is sensitive to unmeasured bias (if the unobserved covariate is dichotomous), ceases to be sensitive if the unobserved covariate is normally distributed. These ideas are illustrated by three examples. It is important to note that the claim in this paper applies to our specific setting of matched pairs with binary treatment and response. Whether the same conclusion holds in other settings is an open question.