I was chatting with some people the other day and the ridiculous voodoo study came up, and that reminded me of an article, “The more you play, the more aggressive you become: A long-term experimental study of cumulative violent video game effects on hostile expectations and aggressive behavior,” published several years ago in the Journal of Experimental Social Psychology.
As we discussed a few years after the paper came out, this article had a huge, huge, HUGE problem, which was that it claimed it was a “long-term experimental study”—that’s right in the title!—but the actual study was not long-term in any way. As I wrote:
What was “long term,” you might wonder? 5 years? 10 years? 20 years? Were violent video games even a “thing” 20 years ago?
Nope. By “long-term” here, the authors mean . . . 3 days.
In addition, the treatment is re-applied each day. So we’re talking about immediate, short-term effects.
I’ve heard of short-term thinking, but this is ridiculous! Especially given that the lag between the experimental manipulation and the outcome measure is, what, 5 minutes? The time lag isn’t stated in the published paper, so we just have to guess.
3 days, 5 minutes, whatever. Either way it’s not in any way “long term.” Unless you’re an amoeba.
Ok, this is not news, indeed it wasn’t even news when I posted on it back in 2018. But it’s still buggin me. As Michael Kinsley said so many years ago, and he was just so so so right on this one, the scandal isn’t what’s illegal, the scandal is what’s legal.
So, the Journal of Experimental Social Psychology published a paper back in 2014 with a blatant error RIGHT IN THE TITLE, and do they retract it? Does anyone even care? No and no.
For your reference, here it is:
Yes, I looked up the erratum listed there, and, no, the erratum does not clarify that the title of the paper is at best extremely misleading and at worst the most horrible thing published in a psychology journal since the critical positivity ratio people dined alone.
But, no, of course nobody would consider doing something about this. It wasn’t noticed by four authors, three peer reviewers, an associate editor, and an editor. Kinda makes you wonder, huh?
I will keep screaming about this sort of thing forever.
P.S. More on this from Kaiser Fung.
If you think that’s bad, check out this publication that claims a “viable alternative to the star building fusion
devices”, powered by a 9-volt battery.
https://www.nature.com/articles/s41545-022-00179-w
Anon:
Big if true. Here’s the first sentence of the abstract: “We present an electro-transmutation (ET) method of water purification using the Quantum Kinetic Fusor™.” Sounds like a joke, but I have no ability to even try to evaluate this claim. It’s interesting to consider this example in light of our recent discussion of that Uber-funded paper. Seems like bad news when a published paper leads off with a trademark!
Yes you do. Just like the average person was able to evaluate the claims of Archimedes by seeing him pull a ship out of the harbor by himself, or those of Newton/Haley/etc by seeing Haley’s comet reappear very close to when predicted.
You can look at the paper and see they are measuring light emissions, mass, and pH after doing something with electrodes to seawater and freshwater. Then you can see there are no theoretical predictions compared to these results, let alone predictions made about future/other data. So there is little reason to accept their electro-transmutation explanation over some other one.
As for whether it replicates (which is tangential to whether the explanation is correct), you could wait to see if others repeat the experiment or try it yourself based on what was described. If it isn’t possible (for someone with the relevant background) to figure out exactly what they did, then you can assume these results are not stable.
Anon:
I agree that we can do this sort of meta-evaluation which does not require deep engagement with the technical claims.
Also, searching around I learned tokamaks seem to require as little as ~1 MW input power, and we know P = I*V. So for a ~10 V battery you would need to discharge at ~100 kA. For comparison, standard 12 V car batteries can discharge at ~1kA.
I’d guess it is possible to construct a 9 V battery with very low resistance that can discharge at 100x the rate of a car battery, at least for a short period of time. It would be easier to increase the voltage 100x, but if it needed to be 9 V for some reason…