Melvyn Weeks writes:

I [Weeks] have a question related to comparability and departures thereof in regression models.

I am familiar with these issues, namely the problems of a lack of complete overlap and imbalance as applied to treatment models where there exists a binary treatment.

However, it strikes me that these issues apply more generally to regression models where there are multiple levels of treatments, and in the limit where the treatment variable is continuous. In this instance a treatment effect is some form of average effect when there is a change of one unit applied to a particular regressor.

Related, just by being able to measure the potential confounding variable and include as a regressor doesnt mean that there are not problems in conducting causal inference from differences in the distribution of regressors differ across many groups. These issues are more apparent in the context of matching when the comparisons are explicit and separate from estimation.

I would welcome your observations on whether the concepts of overlap and imbalance apply more generally to regression models, and problems of undertaking causal inference when there are differences in the distribution of regressors across groups, where the groups may also be individuals in a panel dataset.

My response: Yes, I agree that this can happen with continuous treatments as well. I’m not aware of discussion of this issue in the literature.

Maybe some of you have other thoughts or references?

Look at Mostly Harmless by Angrist and Pischke. They have a section on regression and matching. It is well known that regression estimates can be thought of as weighted averages of covariate-specific effects (i.e. average treatment effect given sex, etc), with weights equal to the conditional variance of treatment given covariates. Relevant section in MH is 3.3.1.

Two useful papers that discuss this issue are:

The Dangers of Extreme Counterfactuals by King and Zeng

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference by Ho, Imai, King and Stuart