A short answer to a short question

Emir Efendic writes:

What is your opinion and can you think of any critiques of the multiple mediation models by Preacher and Hayes (e.g. Preacher & Hayes, 2008)? What would be your method of choice if you were testing multiple possible mediators of an effect, but also if said mediators are connected in a model e.g. X -> M1 which impacts M2 -> Y?

I replied that I have never understood these mediation analyses and I don’t trust them at all! My method of choice is that if I’m interested in many mediators, I’d want to do many different experiments or observational studies: one experiment or observational study for each causal effect of interest.

19 thoughts on “A short answer to a short question

  1. I agree with Andrew that experiments are the way to go. The Preacher and Hayes approach is used widely in studies relying on observational data but most researchers ignore the inherent endogeneity problems when relying on this type of data. 2-stage least squares regression methods are a better alternative favored in econometrics but is not as “point and click” friendly as the PROCESS model advocated by Hayes that has made its way into SPSS and SAS. The difficult part of a 2-stage least squares approach for the social sciences is the identification of instrumental variables. Antonakis and colleagues have a nice primer on the approach but I am not smart enough to fully evaluate it.
    http://www.hec.unil.ch/jantonakis/causality_and_endogeneity_final.pdf

  2. David MacKinnon has used these methods regularly in both experimental and observational contexts. Econometricians use these methods regularly, they just call them autoregressive/autocorrelated models. There are cases where a mediation model is the only true way of representing the data generating process (e.g., “effects of poverty” on educational outcomes), but this seems to be over looked without consideration; in other words, poverty can and never will directly affect a student’s outcomes but can/will affect their access to educational opportunities/materials/supports that will directly affect the outcome.

  3. You might want to read Green, D. P., Ha, S. E., & Bullock, J. G. (2010). Enough already about “black box” experiments: Studying mediation is more difficult than most scholars suppose. THE ANNALS OF THE AMERICAN ACADEMY OF POLITICAL AND SOCIAL SCIENCE, 628(1), 200-208. It is not about multiple mediators per se but is worth a read. I believe that the first author, Donald P. Green, is the same Donald Green involved in the Michael J. LaCour affair.

    • Carol:

      +1 on that article and, yes, this is the same Don Green. In the abstract Green et al. write, “even sophisticated experimental designs cannot speak to questions of mediation without the aid of strong assumptions.” Which I agree with; I’d just want to emphasize that rather than putting in the strong assumptions through a comprehensive model with multiple causal effects on a big multivariate dataset, I’d prefer to attack complex causal questions through a series of separate experiments or observational studies. Even if the same dataset is being mined for the separate studies, I think it can make sense to consider each such study by itself. Ultimately, I think it might be possible to do good inferences using a single comprehensive model but I don’t think we’re there yet, at least not in typical social science problems (sorry Judea!). But I do appreciate the work that Judea and others are doing on this big-picture approach: somebody needs to work on speculative methods of the future, or we’ll never get there.

      • Thank you, Andrew. I now see that you discussed the Green et al. article in your blog post of 7 March 2010.

        One of my concerns is that so many social psychologists use complex methods like this so mindlessly. without really understanding what they are doing. Another is that the reader of articles using such methods has to take too much on faith, because so much of what is going on is hidden.

        • I think social psychologists attempt to answer questions far too ambitious for the experiments they do or the data they collect.

        • I agree, Rahul. I have on my desk an article using a regression model with 76 cases (subjects) and 17 predictors. A second article has 31 cases and 6 predictors.

      • > Even if the same dataset is being mined for the separate studies, I think it can make sense to consider each such study by itself.
        I would second this, understanding “each such study by itself” as each question one is trying to answer given multiple studies.

        Getting something sensible out of multiple observational studies is very challenging given confounding and these issues may well vary by question.

      • Andrew,
        Causal mediation analysis does not rule out running a series of
        separate experiments or observational studies, be it on the
        same dataset or several. It ensures though that we
        explicate ALL the assumptions made in the analysis and
        then, it provides guarantees for the validity of the conclusions.
        It moreover tell us whether the assumptions can
        be relaxed or not. I doubt any of this can be achieved
        without a theory of mediation, but, if you have a way of
        doing it without such theory, the scientific world is eager to listen, and the
        practical world even more so, especially practictioners
        who are uncomfortable with “so much of what is going on is
        hidden” (quoting Carol)

      • “Ultimately, I think it might be possible to do good inferences using a single comprehensive model”
        Are you calling a causal graph a ‘single comprehensive model’? If so, I think you may have the wrong idea. The graph encodes causal assumptions that can be used to justify estimates of causal quantities. It may require fitting many statistical models to estimate the causal quantities of interest. For example, consider using Robins’s g-formula to estimate controlled direct effects (well defined counterfactual quantities that can be consistently estimated under conditions that can be encoded with a causal graph).

        • Z,
          I am not sure if your post was meant for me or for Andrew.
          If for Andrew, I assume he can explain what he means by “do good inferences” — I can’t.
          If for me, then I have a more compelling example to demonstrate that avoiding causal mediation analysis
          would lead one to embarrassment. Consider the notion of indirect effect. You and I know that there is no
          method in the world that would give you the indirect effect through Andrew’s approach of
          “one experiment or observational study for each causal effect of interest..”
          But what about the avoiders? Will they ever understand what they are avoiding?

  4. Emir,
    You might want to read Tyler VanderWeele new book
    “Explanation in Causal Inference” (winner of the 2015 ASA Award)
    which is all about mediation and its many variants.
    Botton line: Mediation analysis has gone through a
    major transformation since 2000, shifting from regression
    to causal analysis. Preacher and Hayes, 2008, as well
    as MacKonnon and Green et al are still stuck in the regression
    age. But David Kenny’s website (2014) has a new section called “causal
    mediation analysis” which recognizes the inevitability of going from regression
    to causation. Causal analysis is what you
    need if you are serious about putting together many different
    experiments or observational studies, as Andrew suggested.
    A gentle introduction can be found here:
    http://bayes.cs.ucla.edu/PRIMER/index.html

    • I’ll be honest, Judea’s idea of a *gentle* introduction differs from mine!

      Morgan and Winship’s “Counterfactuals and Causal Inference: Methods and Principles for Social Research” provides a great overview for social scientists. It draws heavily on Pearl’s work, and relates it to the Rubin / Newman framework.

      • Brian,
        I am very fond of Morgan and Windship (see my blurb on the cover) but, when it comes to mediation analysis, they
        refer to VanderWeele, so I followed suit.
        As for a “gentle” introduction, please tell me if you found any obstacle going cheerfully over the chapters of
        http://bayes.cs.ucla.edu/PRIMER/index.html
        (which, btw, also relates causal inference to the Neyman/Rubin framework, but from a unifying viewpoint (see chapter 4),
        and ends with a toolkit for mediation.)
        So, enjoy the ride, it is really fun.

Leave a Reply to Andrew Cancel reply

Your email address will not be published. Required fields are marked *