In Wooldridge (2005), I posed a problem whose solution involved showing that the so-called ignorability of treatment assumption, commonly used in the treatment effects literature, was necessarily violated. In particular, in a setting with multiple treatment effects and without distributional or functional form assumptions, I assumed that treatment was randomized with respect to the counterfactual outcomes but not with respect to the covariates. I applied the law of iterated expectations to provide a simple proof that ignorability (given the covariates) necessarily fails.
As I stated in the final paragraph of Wooldridge (2005), my result has an obvious implication: one should not entirely rely on “best fit” when selecting covariates for the propensity score (conditional probability of treatment), particularly when some of those covariates might themselves be affected by the treatment. In my example, choosing the model with the best fit would lead to inclusion of covariates that violate ignorability, whereas the proper approach—in the sense that it leads to consistent estimation of the average treatment effect—is to use a constant propensity score (which results in a simple difference-in-means estimate of the average treatment effect).
In the context of a binary treatment, Heckman and Navarro-Lozano (2004) cautioned about the harmful effects of relying on goodness-of-fit measures in matching when some covariates are already known to cause ignorability to be violated. Using simulations with particular functional forms and underlying normality assumptions, Heckman and Navarro-Lozano demonstrate that biases in estimating treatment effects could be larger for matching estimators that employ propensity score models with better treatment predictions. Therefore, Heckman and Navarro-Lozano have priority for making the specific point that maximizing goodness-of-fit can lead to larger biases in estimating treatment effects.
The issue of including covariates that might themselves be affected by treatment dates back to at least Rosenbaum (1984). (I am grateful to Hal White for this reference.) Therefore, I should have cited Rosenbaum's paper, too. For a binary treatment, Rosenbaum characterizes the bias in estimating the average treatment effect when a mediating variable is included among the covariates.
