Michael Sobel sent me this paper which will appear in the Journal of Educational and Behavioral Statistics. It’s about mediation: a crucial issue in causal inference and a difficult issue to think about. The usual rhetorical options here are:

– Blithe acceptance of structural equation models (of the form, “we ran the analysis and found that A mediates the effects of X on Y”)

– Blanket dismissal (of the form, “estimating mediation requires uncheckable assumptions, so we won’t do it”)

– Claims of technological wizardry (of the form, “with our new method you can estimate mediation from observational data”)

For example, in our book, Jennifer and I illustrate that regression estimates of mediation make strong assumptions, and we vaguely suggest that something better might come along. We don’t provide any solutions or even much guidance.

Michael has thought hard about these problems for a long time. (For example, see here and here, or for some laffs, here.) Michael’s also notorious for pointing out that the phrase “causal effect” is redundant: all effects are causal. Anyway, I was interested to see what he has to say about mediation. Here’s the abstract of the paper:

Randomized trials are used to assess the effectiveness of one or more treatments in inducing outcomes of interest. Treatments are typically designed to target key mediating variables that are thought to be causally related to the outcome. Thus, researchers want to know not only if the treatment is effective, but how the mediators affect the outcome. Data from such studies are often analyzed using recursive linear structural equation models, and model coefficients, including the coefficient relating the mediator(s) to the outcome, are endowed with a causal interpretation. However, because only assignment to treatment groups is randomized, not assignment to the mediators, the latter are self selected treatments. In order to believe that the so-called “direct effect” of the mediator on the outcome variable in a structural equation model warrants a causal interpretation one must believe there is no selection bias with respect to the mediator. Holland (1988) studied the case of a single continuous mediator. He criticized the use of structural equation models and showed how to estimate the effect of the mediator on the outcome using treatment assignment as an instrumental variable. However, the assumptions he used to justify the instrumental variable approach are overly strong and substantively implausible. This paper has several goals: 1) to make explicit the assumptions needed to justify equating the parameters of structural equation models with the effects of mediators, 2) to provide weaker and more plausible conditions than those used by Holland under which the instrumental variable estimand may be interpreted as a causal parameter, and 3) to extend the analysis to include the case where subjects do not necessarily take up the treatment to which they are assigned. I also briefly discuss the role of covariates and other possible assumptions to aid in the identification of mediated effects in randomized studies.

Now I just have to read the damn thing…

P.S.

This paper by Heckman and Vytlacil also seems relevant to the discussion.

Andrew,

Thanks for posting this paper by Michael Sobel, for it may serve as a seed for a new discussion. Here are my immediate comments:

1. Michael definitions of direct and indirect effect are identical to mine, in a (2001)

article, by that title , link to: http://ftp.cs.ucla.edu/pub/stat_ser/R273-U.pdf

Yet my conclusions are opposite

to Michael's. I concluded that

structural coefficients are identical to causal coefficients

and, moreover, that formal counterfactual analysis implies

the traditional structural equation rule:

Total effect = direct+indirect effects.

2. The disparity is largely semantics. I define "structural equations" the way they were defined by Haavalmo and other economists before 1965, namely,

as conveyers of counterfactual information. Michael

defines "structural equations"

the way poor econometricians defined it from

1970 till 2000 (the year Heckman

decided to reclaim their counterfactual content).

3. I hope this confusion does not add to the terminological

chaos we already have in causal analysis. But, just in case, let me make a declaration: In my writings

I will consistently refer to

"structural equations" as carrier of counterfactual assumptions. Else, what is "structural" about them?

Why not call them just ordinary algebraic equations, or, as Holland once remarked: "What does Y=ax+e mean? The only meaning

I was able to determine for such an equation is that it is a shorthand way of describing the conditional distribution of

Y given X. (Quoted in causality

p. 137). And as I remarked there, structural equations say NOTHING about the conditional distribution of Y given X.

I surely hope that, by 2009, the counterfactual content of structural equations become universally acceptable.

=======Judea

Mediation is easy.

Andrew, here are my comments regarding your

statements on MEDIATION, (which some call:

EFFECT DECOMPOSITION, or DIRECT AND INDIRECT

EFFECTS).

Gelman writes:

It's about mediation: a crucial issue in causal inference and a difficult issue to think about.

Pearl:

It is an easy issue to think about, if we do

it the way a 12-year old think about it – causally, with no interference from regression addicts.

Gelman:

The usual rhetorical options here are:

– Blithe acceptance of structural equation models (of the form, "we ran the analysis and found that A mediates the effects of X on Y")

Pearl:

I would vow for any word in this blithe acceptance. When my students run the analysis

and claim mediation, I will buy it blindfolded.

I assume your quote meant "we ran thoughtless regression… and found…". Sloppy estimation traditions do not tarnish models. If the structural model claims: total effect = a + bc,

the model is correct, but you have to estimate

the structural parameters, a,b,c correctly,

with assumptions you are willing to defend, and

this might call for more than one-step regression.

Gelman:

– Blanket dismissal (of the form, "estimating mediation requires uncheckable assumptions, so we won't do it")

Pearl:

Every exercise in causal analysis requires unchckable assumptions. We should not do it if we do not have a language to express and judge the

plausibility of our assumptions. We should do it,

if we have such a language.

Gelman:

– Claims of technological wizardry (of the form, "with our new method you can estimate mediation from observational data")

Pearl:

I will buy an ammended version of wizardry:

"with our new method you can estimate mediation from a combination of observational data and vividly stated assumptions".

Gelman:

For example, in our book, Jennifer and I illustrate that regression estimates of mediation make strong assumptions, and we vaguely suggest that something better might come along. We don't provide any solutions or even much guidance.

Pearl:

Something better did come along: weaker conditions (read: vivid assumptions) for consistent estimation of mediation, not necessarily through a one-step regression

So, Aristotle was right: Causality is easy,

if we only learn to do it the way children do it,

before the age of regression addicts.

======Judea

This area seems to be getting a lot of attention recently. I wonder what you make of Kosuke Imai, Luke Keele, Dustin Tingley, and Teppei Yamamoto's work. They seem to say that demonstrating causation is difficult, but not impossible: http://imai.princeton.edu/projects/mechanisms.htm…

"An important goal of social science research is the analysis of causal mechanisms. A common framework for the statistical analysis of mechanisms has been mediation analysis, routinely conducted by applied researchers in a variety of disciplines including epidemiology, political science, psychology, and sociology. The goal of such an analysis is to investigate alternative causal mechanisms by examining the roles of intermediate variables that lie in the causal path between the treatment and outcome variables. In this collection of papers we advance the statistical analysis and experimental design of causal mechanisms in several important ways. 1) We formalize mediation analysis in terms of the well established potential outcome framework for causal inference. 2) We introduce a minimal set of assumptions thatidentify the causal mediation effects. 3) We show how to conduct sensitivity analyses to violations of this identifying assumption. Our sensitivity analysis allows researchers to ask, how large a violation would be necessary before their results would be reversed. 4) We extend our proposed methods to various types of data and statistical models. Our method can accommodate linear and nonlinear relationships, parametric and nonparametric models, continuous and discrete mediators, and different types of outcome variables. 5) We show how to design randomized experiments in order to identify causal mechanisms. 6) We provide an easy to use package in the free software language R that implements everything discussed in the papers."