Adjudicating between alternative interpretations of a statistical interaction?

Jacob Felson writes:

Say we have a statistically significant interaction in non-experimental data between two continuous predictors, X and Z and it is unclear which variable is primarily a cause and which variable is primarily a moderator. One person might find it more plausible to think of X as a cause and Z as a moderator and another person may think the reverse more plausible. My question then is whether there is are any set of rules or heuristics you could recommend to help adjudicate between alternate perspectives on such an interaction term.

My reply:

I think in this setting, it would make sense to think about different interventions, some of which affect X, others of which affect Z, others of which affect both, and go from there. Rather than trying to isolate a single causal path, consider different cases of forward casual inference. My guess is that the different stories regarding moderators etc. could motivate different thought experiments (and, ultimately, different observational studies) regarding different potential interventions.

So I would not try to “adjudicate” between different stories; rather, I’d recognize that they could all be appropriate, just corresponding to different interventions. Also, all the above would hold even if there are only main effects, no interactions needed. And, for that matter, statistical significance would not be needed either for you to look at these questions.

3 thoughts on “Adjudicating between alternative interpretations of a statistical interaction?

  1. Jacob Felson:

    I can’t add much to Andrew’s answer regarding heuristics to identify which predictor is the primary cause and which one is only the moderator. I think there is no way to identify the direction of causality between variables if you only have cross-sectional data in your Data Analysis, so there can be no heuristic. This applies not only to independent vs. dependent variables but also towards interactions. If you had panel data, there would be some options but otherwise you will have to resort to theoretical (causal) reasoning.

    Maybe you already know them, but the papers from Brambor et al. (2006) and Berry et al. (2012) are probably good to know if you work with interaction models (they are written from a classical frequentist viewpoint, though). While Brambor et al. mostly help preventing some typical mistakes in evaluating models with interactions, Berry et al. hint that you might also want to look at the specific changes in marginal effects of both X and Z. These might even help you to see if your data really plausibly fit together with the “causal stories”, Andrew advises to think about. There’s still no direct heuristic to discern between moderators and causal predictors but using more of the information you have in your data might still be useful. Have a look how the effect of X varies by Z exactly and how the effect of Z varies by X. Does the sign of the effect change? If not: Does the effect only increase from mediocre to strong or does it change from nearly no effect to a meaningful effect size? To make things even more complicate, you really might want to consider non-linear effects of your predictors because ignoring them may distort the estimation of your interaction coefficients dramatically. You probably know most – if not all – of that already but I’ve seen so many wrongly interpreted interaction models in published papers in sociology in the last time, that I feel the obligation to mention these issues all the time when interactions come up.

    Berry, William D., Matt Golder, und Daniel Milton. „Improving tests of theories positing interaction“. Journal of Politics 74, Nr. 3 (2012): 653–71.
    Brambor, Thomas, William Roberts Clark, und Matt Golder. „Understanding interaction models: Improving empirical analyses“. Political analysis 14, Nr. 1 (2006): 63–82.

    • Sanjay Srivastava:

      That’s what I tried to aim at when I mentioned that you can maybe do more when you have panel data. If the time order *is* clear, I think I agree with Kraemer et al. Usually we don’t have such a clear time order, especially not with most of the socia science data. That’s a problem and we probably should try to collect far more panel data but for now we have to deal with it.

      Thank you for mentioning the Kraemer et al. paper, I’ve heard of it but so far neglected to read it.

Comments are closed.