This post is by Lizzie. The Figure is take from Frank 2024.
I was in a meeting a little over a year ago in which I asked a student to define causal inference. The definition he gave me focused on complex approaches often used to try drag causality out of observational data. So I asked if causal inference involved experiments at all? “No,” came the reply. I double-checked. “No.” The student was certain. Someone else following up later did not change their mind.
Experiments cannot help with causal inference.
I knew we had a problem then, but how did it happen? I’ll tell you my version of what happened and some of what I can put together for how this happened, but I am open to other theories and ideas. And if perhaps the new ‘causal inference’ movement in ecology really has — finally — struck on a way for us to figure out ecology, then time will obviously prove me wrong, and you’re welcome to beat time to it in the comments section.
I could start back with Sewall Wright and the demes of cows I was once told he used to visit and Fisher and his fields of corn (or some agreeable consistent crop just waiting for its split plot design), but I will just start in the 1990s with path analysis in ecology. Path analysis (what I would call structural equation modeling with standardized coefficients) was hot in the 1990s in ecology. There was a chapter on it by Mitchell in the book ‘Design and Analysis of Ecological Experiments’ in 2001 (perhaps around its peak). It had this on the first page:
plant traits → visitation → pollination → reproduction
Isn’t that great? I could link plant traits to plant reproduction via those traits’ effects on (insect) visitation (to flowers) and how all that racey visiting led to pollination and then — reproduction (and then I might even make a run at … plant fitness!). I mean it is great. I like the idea. I liked the chapter. I did a path analysis. But I didn’t call it path analysis, I called it structural equation modeling because, by the time I was publishing, path analysis had hit some bumps.
Namely, everyone had done path analysis and many of those people had done it poorly in one way or another and suddenly all those little paths looked like a lot of made up stories with lots of little asterisks representing lots of significant p-values that didn’t really hold up to scrutiny. Shocker! (No, not shocker.) So, we all stopped doing path analysis and (within a few years it seems to me) we started doing structural equation modeling, sometimes with standardized coefficients. But we never called it path analysis again.
We couldn’t let go of path analysis because the dream was still alive. We wanted causality. We wanted to link things to explain how the world works. And manipulating plant traits is hard (have you ever tried to paint flowers different colors in a field? Or paste on tiny hairs (which we call trichomes)?), but measuring them is comparatively less hard. We wanted causality from observational data. That was the dream.
And, the dream is still alive. After all this time.
And the dreamers seem to have just discovered some of the basics of causal inference for observational data from the social sciences and econometrics literatures. With this, they have discovered that diversity (more species) in grasslands leads to lower productivity, not higher (Dee et al. 2023) and linked white nose syndrome in bats to increased infant mortality across the eastern US (Frank 2024). This latter paper is the one that rattled me because I attended a discussion group with colleagues and found out how many of my colleagues are excited by these ‘new techniques’ and how they have learned from them the amazing power of fixed effects for finding causality and the dangers of random effects to lead us astray.
Huh? Fixed effects to save the day and random effects of doom?
I tracked some of this down to me thinking of the common ecology definition of fixed versus random (I think some closer to definition #2, page 245 of Gelman and Hill: “2. Effects are fixed if they are interesting in themselves or random if there is interest in the underlying population. Searle, Casella, and McCulloch (1992, section 1.4) explore this distinction in depth.”) whereas the ‘new’ methods in ecology are using (I believe) definition # 5 (“5. Fixed effects are estimated using least squares (or, more generally, maximum likelihood) and random effects are estimated with shrinkage (“linear unbiased prediction” in the terminology of Robinson, 1991). This definition is standard in the multilevel modeling literature (see, for example, Snijders and Bosker, 1999, section 4.2) and in econometrics….”).
This explains some of the interesting lines I found in these papers, including:
Random effects account for clustering in data via the error structure of the model (Bolker et al. 2009; Gelman and Hill 2006), rather than estimating cluster means as part of the data generating process of a model (i.e., via fixed effect for each cluster’s mean, using the terminology of the mixed models literature). (Byrnes & Dee 2025)
The time-varying site attributes (μ_{st}) are also modeled in a fully flexible way that allows a year- specific effect for each site (in the estimation, an indicator for each year is interacted with an indicator for each site). (Dee et al. 2023)
I think the authors of this new Tower of Babel for ecology have also defined random and mixed effects to mean only ever linear models with lmer-style partial pooling on intercepts (never slopes I presume?) with fixed effects on slopes (back to definition #2). They even go so far as to refer to this as the “Common Design in Ecology” (they also capitalize Ecology and Ecologists in Byrnes & Dee 2025, which I find odd — is this high German? Personally, as an ecologist, I don’t think I need an capital letter) and explain:
Without more variable transformations, the multi-level modeling approach does not easily lend itself to controlling for as many unobservable sources of confounding as can be done in our linear, additive, fixed-effects panel data estimator. (Dee et al. 2023, in supp)
I thought about calling this post ‘The Tower of Babeling Causality’ or ‘The Problem with Statistical Terminology,’ but the real problem is not how lost in the weeds of words we get in with terminology. It’s partly how easily ecologists do want ‘new’ terms and approaches that will solve everything. The authors who have come armed with econometrics panel data approaches and instrument variable analysis (when most ecologists don’t know what is an instrument in their experiments) had the ground laid for them by all the ecologists who are enthralled by ‘random effects.’ I agree we have too many people trained to believe that chucking enough categorical covariates (site, plot, year …) on the intercept of a simple linear model will save the day. It’s a problem how much we sway from this being correct statistics to that being correct statistics. And how quickly think a new approach will change everything in ecology. We seem to quickly learn — and re-learn — that bad stats can easily lead you astray, but never take on that good statistics alone will not save you.
To be clear, I don’t have a giant problem with these methods. I have a problem with how they are presented as saviors (and somewhat how they are presented as new, but perhaps we need the ‘new’ and ‘savior’ angle to follow Grace) but I have a bigger problem in how rapidly they are being taken up. I fear the next 10 years I will live in a sea of piranhas where lots of ecological problems explain 5-10% of infant mortality and plant productivity.
And that’s the other problem — the bigger one: how much people want this causality. They want to believe that we have the data and methods to show that a disease that wipes out bats leads to an 8% increase in infant mortality. Of course we should want causality, we’re scientists, but the drive for causality seems to jettison a lot of the stuff we also need as scientists, especially estimates of uncertainty and the ability to leave room for uncertainty so that we search out better methods and better answers. I don’t know if bat decline has increased infant mortality 8% (though I highly doubt that number given the language of the author and how ‘outrageous’ he thinks it is that he is expected to share all his data for people to believe his claims). I just know we have managed to do science before and make progress and it wasn’t because we got better statistical methods or memorized a glossary of one particular set of people’s definitions of DAGs and fixed effects.
I am cited in one of these papers for old work I did where I compared shifts in the timing of flowering and leafout with warming over time (due to anthropogenic climate change and natural variation) and experimental warming (due to infrared heaters or teeny tiny plastic greenhouses — also, hello instruments in ecological experiments!). Estimates from experimental and observational data were different — the effect of warming in observational data was bigger. I did lots of different statistical analyses to figure this out, I even did something probably close to the ‘Common Design in Ecology’ (although the authors don’t seem to ding me for this) and the effect never went away. With Ailene Ettinger and other colleagues, I eventually got all new data and found the same thing using slightly different statistics. But that wasn’t why we got all the new data, we got it to test hypotheses about what drove the difference. And we found out that it appeared to be two things: warming experiments dry out soils which delays leafout and flowering and warming experiments over-report their warming (so their per degree estimates look smaller than they should).
I did all of this without ever invoking the term ‘causal inference.’ And that’s what really worries me for trainees today; that ‘causal inference’ will now mean a narrow branch of amazing ‘fully-flexible’ completely un-confounded statistics. We’re ecologists; we actually can manipulate some stuff. And somehow we’re going so gaga for econometrics statistics to give us causality through time-invariant fixed effects (or whatever) that we have students who don’t know how experiments could relate to causal inference.
What’s the solution? If you ask me, be less gaga over any statistical method (and I do love my own statistical methods so I could practice a little more of what I preach) and teach everyone basic mathematical notation and basic biological models. Teach them that generative modeling doesn’t belong to any one part of statistics or to only fixed or random effects. Teach them to be able to write out a simple biological model and simulate data from it and then fit their statistical model to it. ‘Only connect!’ Connect the models you learn for ecological theory with those you learn in stats. (And maybe teach them about the long debate in conservation biology about Cassandra’s curse, but that is a topic for another post.)
