Do not control for post-treatment variables?

David Afshartous writes:

Regarding why one should not control for post-treatment variables (p.189, Data
Analysis Using Regression and Multilevel/Hierarchical Models), the argument is very clear as shown in Figure 9.13, i.e., we would be comparing units that are not comparable as can be seen by looking at potential outcomes z^0 and z^1 which can never both be observed. How would you respond to someone that says “well, what about a cross-over experiment”, wouldn’t it be okay for that case?” I suppose one could reply that in a cross-over we do not have z^0 and z^1 in a strict sense, since we observe the effect of T=0 and T=1 on z at different times rather than the counterfactual for an identical time point, etc. Would you add anything further?

My reply: it could be ok, it depends on the context. One point that Rubin has made repeatedly over the past few decades is that inference depends on a model. With a clean, completely randomized design, you don’t need much model to get inferences. A crossover design is more complicated. If you make some assumptions about how the treatment at time 1 affects the outcome after time 2, then you can go from there.

To put it another way, the full Bayesian analysis always conditions on all information. Whether this looks like “controlling” for an x-variable, in a regression sense, depends on the model that you’re using.

1 thought on “Do not control for post-treatment variables?

  1. one assumption of cross-over design is no carry-over effect. a washout period is usually included between the 2 consecutive treatment periods to ensure the assumption. I cannot image we can use such a design if the assumption is violated in a clinically meaningful way.

Comments are closed.