Lizzie told me about this paper, “Bidirectionality, Mediation, and Moderation of Metaphorical Effects: The Embodiment of Social Suspicion and Fishy Smells,” which reports:
As expected (see Figure 1), participants who were exposed to incidental fishy smells invested less money (M = $2.53, SD = $0.93) than those who were exposed to odorless water (M = $3.34, SD = $1.02), planned contrast i(42) = 2.07, p = .05, Cohen’s d = 0.83, or fart spray (M = $3.38, SD = $1.23), i(42) = 2.22, p = .03, d = 0.78.
Fart spray!
The paper faithfully follows Swann’s 18 rules for success in social priming research.
I was surprised to see that people were still doing this sort of thing . . . but then I looked at the paper more carefully. Journal of Personality and Social Psychology, 2012. Just one year after they’d published that ESP paper. To criticize a psychology journal for publishing this sort of thing in 2012 would be like mocking someone for sporting a mullet in the 1980s.
Of course, just cos a paper is on a funny topic and just cos it follows the cargo-cult-science template, it doesn’t mean that it is wrong. I guess I’ll believe it when I see a preregistered replication, not before. In the meantime, just recall that experimental results can be statistically significant and look super-clean but still not replicate. The garden of forking paths is not just a slogan, it’s a real thing that can easily lead researchers to fool themselves; hence the need to be careful.
P.S. All that said, it’s still not as bad as “Low glucose relates to greater aggression in married couples”: that’s the study where they had people blasting their spouses with loud noises and sticking pins into voodoo dolls.
These issues also arise with published research on more important topics.
“Forty-five students (mean age 20.1 years, 22 women) at the University of Michigan participated in a one-shot trust game…each received 20 quarters ($5) and an investment form with instructions. ”
oops. call retraction watch.
But as Andrew alludes, I think the give-a-student-a-quarter-and-write-a-paper-about-what-happens experimental procedure has mostly died.
If you give a mouse a cookie… if you give a student a quarter….
So, Andrew…did you have a mullet?
Yes. But it doesn’t really show up in slide 6 here.
It was a long-tailed distribution.
Groan!
There was something fishy about that mullet remark.
Microgroan