Dan Kahan writes:

We all know it’s ridiculous to be able to go on an fMRI fishing trip & resort to post hoc story-telling to explain the “significant” correlations one (inevitably) observes (good fMRI studies *don’t* do this; only bad ones do– to the injury of the reputation of all the scholars doing good studies of this kind). But now one doesn’t even need correlations that support the post-hoc inferences one is drawing.

This one’s good. Kahan continues:

Headline:
Religious Experiences Shrink Part of the Brain

text: ” … The study, published March 30 [2011] in PLoS One, showed greater atrophy in the hippocampus in individuals who identify with specific religious groups as well as those with no religious affiliation … The results showed significantly greater hippocampal atrophy in individuals reporting a life-changing religious experience. In addition, they found significantly greater hippocampal atrophy among born-again Protestants, Catholics, and those with no religious affiliation, compared with Protestants not identifying as born-again.”

Waaaaiiiit!

I [Kahan] thought: *must* be reporter’s misunderstanding. But the article language actually tracks the abstract: “Significantly greater hippocampal atrophy was also observed from baseline to final assessment among born-again Protestants, Catholics, and those with no religious affiliation, compared with Protestants not identifying as born-again.” And this in text: “Table 2 presents longitudinal regression models of religious factors and covariates on change in left and right hippocampal volumes. Positive model coefficients indicate less atrophy over time. Reported life-changing religious experience at baseline was associated with greater atrophy between baseline and follow-up in the left and right hippocampus (left: b = −0.45, P<.001; right: b = −0.32, P = .012). Born-again Protestant group membership at baseline was associated with greater atrophy in the left and right hippocampus compared with non born-again Protestant group membership (left: b = −0.15, P = .046; right: b = −0.15, P = .050). Catholic group membership (n = 22) (b = −0.22, P = .046) and no religious group membership at baseline (n = 19) (b = −0.28, P = .046) were also associated with greater atrophy in the left hippocampus over time compared with non born-again Protestant group membership..." And the authors really do *say* that these patterns support an inference that religious experiences affect the brain: "The findings of this study indicate that certain religious factors may influence longitudinal change in hippocampal volume during late life. Greater hippocampal atrophy over time was predicted by baseline identification as born-again Protestants, Catholics, or no religious affiliation, compared with Protestants who were not born-again..."

You might wonder why Kahan is going to the trouble to tell me about a botched research report that made its way onto the pages of Scientific American. Any individual case of bad science is surprising, but once it’s uncovered, it’s just one more ugly wreck on the side of the road.

Kahan explains that he had a therapeutic motivation:

Anyway, thought focusing on this would help get your frustration about persistence of “what’s matter w/ Kansas” fallacy out of your system [see here and here], since it’s impossible to be insanely frustrated about 2 things at once.

And, as a bonus:

But also, so you get over being insanely frustrated at all, consider how satisfying it can be to consider a valid & interesting piece of research: Weisberg, D.S., Keil, F.C., Goodstein, J., Rawson, E. & Gray, J.R. The seductive allure of neuroscience explanations. J. Cognitive Neuroscience 20, 470-477 (2008), which shows that people will accept fallacious inferences more readily when told that those inferences are supported by fMRI images.