“Analysis challenges slew of studies claiming ocean acidification alters fish behavior”

Lizzie Wolkovich writes:

Here’s an interesting new paper in climate change ecology that states, “Using data simulations, we additionally show that the large effect sizes and small within-group variances that have been reported in several previous studies are highly improbable.”

I [Lizzie] wish I were more surprised, but mostly I was impressed they did the expensive work to try to replicate these other studies and somehow managed to get it published, given that “[t]he research community in the field of ocean acidification and coral reef fish behaviour has remained small.”

And here’s a news report with more background.

I’ve heard from some very important Harvard professors that the replication rate in psychology is quite high—indeed, it is statistically indistinguishable from 100%. I guess it’s a bit lower in other fields. We should all aspire to the greatness that is Ivy League psychology.

5 thoughts on ““Analysis challenges slew of studies claiming ocean acidification alters fish behavior”

  1. Interesting. I know essentially nothing about these types of studies, nor the properties of these data, but I do notice in the supplement the following:

    “However, three limitations prevented us from analysing the data over time. First, the effect of time was nonlinear. Second, the data were temporally auto-correlated. Third, the data were bimodal around the minimum and maximum values (see Extended Data Fig. 3 for an example) and did not conform to any distribution readily available for use in generalized additive mixed models (with the mgcv package in R)”

    I have no idea in which direction these might bias the analyses, bu usually these can be pretty important factors to ignore.

      • There are many quite simple ways to include nonlinear (in fact non-parametric) time trends in a time series analysis. In fact, I do believe mgcv has them. The whole literature on state-space decomposition (Kitagawa, Gersch, Harvey, Koopman etc) is about that. I believe INLA can handle a lot of these issues, though INLA can be a bear to learn to use.

    • Sounds like they need a good time series analyst. Effects across time are very often nonlinear (e.g. they might asymptote).

      The fact that the data are bimodal makes me think that there might be some outlier editing going on in one or the other study — and outlier editing can account for spurious results one way or the other. But, like Roy, the closest I’ve come to this type of study is to occasionally eat a fish.

Leave a Reply to Joshua Cancel reply

Your email address will not be published. Required fields are marked *