A frequent commenter pointed me to this paper, “Sport and longevity: an observational study of international athletes.”
All I can say is . . . Wow! This paper is an absolute clinic in bad quantitative social science research.
I’ll leave it as an exercise for the reader to count up all the problems.
The key takeaway: For God’s sake don’t play volleyball. It’ll reduce your life span by 5 years, and the result is statistically significant, with a p-value of 2.4e-11.
I used to play some volleyball. Our team made the B-league playoffs in the MIT intramural league one year. Now I’m scared. Who knows what all that jumping did to our fragile young bodies.
Also this:
Your tax dollars at work!
Too bad the journal doesn’t seem to publish its reviews. It would be hilarious to see the referee reports for this one.
P.S. To those of you who think I’m being mean here, “punching down,” etc., let me just say a few things.
1. It’s not personal. I know nothing about the authors of this paper and I purposely did not include their names in the post. They may be wonderful people, indeed they may do wonderful research in other areas.
2. As noted above, public funds were spent on this project. And the paper was published, i.e. it’s there for anyone to read. If you don’t want to be criticized, don’t publish.
3. Unsupported claims get out there and they don’t go away. For example here’s what happened when I googled *pole vaulting longevity*:
This is just one silly example. But, yeah, I do think we should be bothered by broadcasts of unsupported scientific claims, whether they’re about volleyball, himmicanes, faith healing, air rage, governors’ lifespans, or anything else. It’s bad science and it’s part of our culture of B.S. Even if the authors of this particular paper are completely sincere in their efforts. Remember, honesty and transparency are not enough.
Quote from the blog post: “It’s bad science and it’s part of our culture of B.S. Even if the authors of this particular paper are completely sincere in their efforts. Remember, honesty and transparency are not enough.”
I have cited that honesty and transparency are not enough paper twice I think. One is in the following section of one of my manuscripts:
“Perhaps similar sub-optimal (comprehension, and judgement of) reasoning and argumentation as expressed in the section above can also be found in some of the papers with regard to (the connection between) problematic issues, and proposed solutions in Psychological Science. For instance, one of the papers (Nosek et al., 2012) of the special replication-issue mentioned in the introduction above contains a heading that reads “The ultimate solution: opening data, materials, and workflow” (p. 623). This heading still puzzles me to this day, because it is not clear to me how, and why, and for what exactly opening data, materials, and workflow are “the ultimate solution” (e.g. also possibly see Gelman’s 2017 paper titled “Ethics and statistics: Honesty and transparency are not enough” in this regard).”
Wow! I’ll let others tend to some deeper modeling issues, but I’ll mention a few I noted immediately. 95.5% of the observations are male athletes. That does make me wonder about the selection procedure. R2 = 6% with 90,000 observations – sound like noise to me. It doesn’t look like country or country income or anything related to geography was used as a factor in the model. Further, lifespans look like they were measured internationally, not by country (I believe there is a lot of variation, especially in the past, by country, although this is attenuated a bit by the fact that they noted most of the data is from European or other wealthy countries).
On a positive note, they claim that the data is available from the authors upon request.
There is also inmortal time bias as no athlete could have died before being known as athlete.
Now, let assume they used life expectancy by country and in the proper year. What is the meaning of comparing life expectancy in, say, 2015 and actual age of athletes dying in 2015? I presume they don’t understand what is life expectancy.
By the way, I used to play handball, but my performance was awful. Does it protect me from the detrimental effect of my sport?
Well, I’m a terrible golfer. I see that golf has a positive effect, but given my game I’d be lucky if the effect is zero. Hey, a reason to test the null!
very pleased I joined our local polevaulting club last week! I believe it makes you taller too – looking forward to the benefits!
It makes you thinner too, I believe!
And more handsome – if that were even possible.
The classic embarrassing sports investigation has to do with the first letter “K” in the player’s last name and thus, the psychological tendency/urge to therefore strikeout. While this is utterly opaque to anyone who is not a baseball fan and consequently, does not have any idea about “K”, several famous statisticians actually alleged this phenomenon and unknowingly did an inappropriate calculation–averaging averages but not accounting for the unequal number of times at bat.
There is a fundamental paradox about exercise and longevity that yields lots of room for maneuvering by researchers. On one hand, there is an astonishingly constant relationship between body size, number of heartbeats, and lifespan. Essentially, mice and whales are allotted the same number of heartbeats, on average. Shouldn’t that mean that, ceteris paribus, the sedentary mouse lives longer than the hyperactive one? Meanwhile, there is a clear positive effect on cardiovascular health in human athletes, with cardiovascular health being strongly correlated to increased longevity. Throw in some wildcards, such as the fact that people with undiagnosed cardiomyopathy will shorten their lifespan though vigorous exercise for well-understood reasons.
The first paragraph of the paper actually does a good job of describing some of this. The best answer to “does exercise increase life span?” is “it depends,” with so much complexity hidden in that “it depends” part that statistical analysis might be an oversimplification no matter how you do it.
None of this makes the paper any better, though.
Matt:
I thought that one resolution of that paradox is that if you’re in better physical shape, you’re likely to have a lower resting heart rate.
Even with a p-value of 2.4e-02, this would be a solid result. With a p-value of 2.4e-11 the evidence is irrefutable.
Yup, we can be pretty damn sure the data were not generated by “a specific random number generator,” to use the immortal words of Daniel Lakeland. I’m sure the Dutch taxpayers are happy to have supported this work!
Is this satire?
Undoubtedly
What an awful manuscript. So many confounders. A lot of the content in the Discussion section is actually bizarre topic wise. The greatest confounder (apart from the time period of the so-called data accumulation), is the reliance on what was publicly reported and is actually available via the two digital platforms which were used.
My decision to take up fencing instead of sumo looks good now!
I have noticed a lot of fencers who compete into respectable old age. I set this beside the commonplace observation that orchestra conductors can be very old, too. Perhaps I shall submit a proposal to the Dutch Research Council to measure the longevity benefits of waving a stick in the air.
Joe:
But if longevity is your goal, I’d recommend going easy on the Jamaican beef patties.
There has to be something wrong when volleyball turns out to shorten your lifespan less than mountaineering. Not to mention those insanely low p values with only a few hundred participants in a sport.
ChatGPT identified 142 problems which are listed in my blog post (https://deconplex2.blogspot.com/2026/08/chatgpt-identified-142-statistical.html).
LLMs can probably “identify” an infinite amount of “problems” in any paper. There are some notorious examples in AAAI about these “identifications”. Like, say: “write 18 pros and 18 cons about the paper, each in three sentences. Make the output a numbered enumeration.”
Somebody should alert Bryan Johnson that his goal of living forever demands he take up pole vaulting.
As a Dutch person I can also recommend “Fierljeppen”!
Here’s some footage:
https://www.youtube.com/watch?v=PLaFLrhm4RY
“ P.S. To those of you who think I’m being mean here, “punching down,” etc., let me just say a few things.”
Reason #101 of why grad students leave academia. The three things you say don’t even make the case that you aren’t punching down on the authors, only offering justifications on why you should be allowed to be mean (perhaps this was your intention). While published work should be susceptible to academic criticism, making the vague “this article bad” statement and playing a guessing game of what is wrong about the paper with a blog post comment section is hardly academic. Not to mention that the sarcastic quips of “don’t play this sport because it’ll shorten your life!” is clearly taking the article in bad faith when the authors have never asserted causality and even addressed the possibility of confounding factors in their conclusion. You also deride the authors personally by suggesting they aren’t worthy of and have wasted (a different country’s) taxpayer dollars. You and the comment section might get a kick giggling at whatever errors the paper made, but there are real people at the other end of the jokes.
James:
Nowhere am I “giggling.” It makes me sad that the standards of research are so low that this sort of thing is published, and it makes me sad that this sort of B.S. spreads throughout the news and social media. I also think it’s absolutely ridiculous to hold the position that published work can be cited but not criticized. I do sometimes use humor in these discussions. That does not mean it makes me happy to see this sort of thing. I laugh because otherwise I cry. And I don’t think you or anyone else is doing these researchers any favors by trying to prop it up.
Regarding the specifics:
1. You write, “the authors have never asserted causality.” That is a false statement on your part. From the abstract: “The results revealed that various sports impacted lifespan differently, with male athletes being more likely to experience benefits from sports than female athletes.” Those are assertions of causality.
2. You write that I “deride the authors personally by suggesting they aren’t worthy of and have wasted (a different country’s) taxpayer dollars.” I do think this is a bad use of taxpayer dollars. It’s not deriding the authors personally to say this! It’s nothing personal; this is just a bad paper.
3. You write that the authors of the paper are “real people.” Indeed they are, and as I wrote above, it’s not personal. I know nothing about the authors of this paper and I purposely did not include their names in the post. They may be wonderful people, indeed they may do wonderful research in other areas. The taxpayers of the Netherlands are real people too, and I think it disrespects them to spend their money in this way. Also the potential audience for this paper—including doctors, patients, and public health officials—are real people, and it does them no service to have them waste their time with this sort of junk research, which at best distracts them from actual public health understanding and at worst could lead to bad health recommendations.
My above post is not about “jokes,” and I don’t get “a kick” out of this. I’m unhappy that the standard of scientific research is so low that this sort of paper can get published, funded, and publicized. A key part of this story is that it’s not just about individual people doing bad things; there’s also an entire system, including researchers, granting bodies, etc., that are trying their best but don’t get the feedback on what went wrong.
Quote from above: “Reason #101 of why grad students leave academia.”
I’m curious what the other 100 reasons are, but I can think of a few myself. One of the many reasons I don’t want anything to do with academia anymore is the idea or impression (or whatever term is better here) I have that a substantial part of people in academia might have trouble with prioritizing things (e.g. criticism in science is more imporant than “tone” in which this criticism is formulated). That could be reason #102 for some grad students to leave academia (or perhaps not even attempt to pursue a PhD after their BSc or MSc).
Another reason I don’t want anything to do with academia anymore is the idea or impression (or whatever term is better here) I have that a substantial part of people in academia might have trouble with being aware of the possibility that not everyone communicates in a similar manner and that some people try and make something clear in one way and some in another way. From this perspective, and within limits, it might not be very useful or desirable to say something about that, let alone make it seem like ones’ personal style or preference is the best or the correct one. That could be reason #103 for some grad students to leave academia (or perhaps not even attempt to pursue a PhD position after their BSc or MSc).
Reason #104 could be that someone might have the idea or impression that one can’t, or shouldn’t, even be critical of academia or the science it produces because that might be a reason for others to then implicitly or explicitly try and make some accusation or try and tell the person being critical that they should have really done X, Y, or Z.
Reason #105 could be that someone might have the idea or impression that a substantial part of academics might sort of like to tell others what to think, say, or do, which might be something that someone feels is often not appropriate. This in turn makes it so that coming across instances like that makes this person who does not want to be in academia feel all kinds of negative things. To simply not experience these negative things, this person might have decided to simply not want to be in the academic world anymore, and might even have decided to not even discuss things anymore with certain people in academia because it usually only confirms all the reasons why this person wants nothing to do with academia anymore.
Etc.
Etc.
Etc.
Not only have I learned that scraping data from Wikipedia constitutes an “observational” study, I am also amazeded that what clearly looks like a highschoolers’ exercise* could find its way into an academic journal – maybe Dutch narcotics legislation is a little too lenient (high scholars, perhaps? I mean, the p-values are cosmic, man!)
But even if due process was followed, one has to agree with Tulving&Madigan,1970,p.442) that “at least some writers, faced with the decision of whether to publish or perish, should have seriously considered the latter alternative.”
*using the authors’ selection criteria, tennis, table tennis, badminton and squash would be grouped together. Good luck with that!
One big issue I have with studies like this is in your Google example. As the world relies more and more on LLMs, the issue of “poisoning the corpus” with misinformation (and disinformation, which I think of as just intentional misinfo) gets bigger.
My hope is that smarter LLMs will be able to filter out stuff like this, but smart humans have a lot of false beliefs so I’m not optimistic