A frequent commenter pointed me to this paper, “Sport and longevity: an observational study of international athletes.”
All I can say is . . . Wow! This paper is an absolute clinic in bad quantitative social science research.
I’ll leave it as an exercise for the reader to count up all the problems.
The key takeaway: For God’s sake don’t play volleyball. It’ll reduce your life span by 5 years, and the result is statistically significant, with a p-value of 2.4e-11.
I used to play some volleyball. Our team made the B-league playoffs in the MIT intramural league one year. Now I’m scared. Who knows what all that jumping did to our fragile young bodies.
Also this:
Your tax dollars at work!
Too bad the journal doesn’t seem to publish its reviews. It would be hilarious to see the referee reports for this one.
P.S. To those of you who think I’m being mean here, “punching down,” etc., let me just say a few things.
1. It’s not personal. I know nothing about the authors of this paper and I purposely did not include their names in the post. They may be wonderful people, indeed they may do wonderful research in other areas.
2. As noted above, public funds were spent on this project. And the paper was published, i.e. it’s there for anyone to read. If you don’t want to be criticized, don’t publish.
3. Unsupported claims get out there and they don’t go away. For example here’s what happened when I googled *pole vaulting longevity*:
This is just one silly example. But, yeah, I do think we should be bothered by broadcasts of unsupported scientific claims, whether they’re about volleyball, himmicanes, faith healing, air rage, governors’ lifespans, or anything else. It’s bad science and it’s part of our culture of B.S. Even if the authors of this particular paper are completely sincere in their efforts. Remember, honesty and transparency are not enough.
Quote from the blog post: “It’s bad science and it’s part of our culture of B.S. Even if the authors of this particular paper are completely sincere in their efforts. Remember, honesty and transparency are not enough.”
I have cited that honesty and transparency are not enough paper twice I think. One is in the following section of one of my manuscripts:
“Perhaps similar sub-optimal (comprehension, and judgement of) reasoning and argumentation as expressed in the section above can also be found in some of the papers with regard to (the connection between) problematic issues, and proposed solutions in Psychological Science. For instance, one of the papers (Nosek et al., 2012) of the special replication-issue mentioned in the introduction above contains a heading that reads “The ultimate solution: opening data, materials, and workflow” (p. 623). This heading still puzzles me to this day, because it is not clear to me how, and why, and for what exactly opening data, materials, and workflow are “the ultimate solution” (e.g. also possibly see Gelman’s 2017 paper titled “Ethics and statistics: Honesty and transparency are not enough” in this regard).”
Wow! I’ll let others tend to some deeper modeling issues, but I’ll mention a few I noted immediately. 95.5% of the observations are male athletes. That does make me wonder about the selection procedure. R2 = 6% with 90,000 observations – sound like noise to me. It doesn’t look like country or country income or anything related to geography was used as a factor in the model. Further, lifespans look like they were measured internationally, not by country (I believe there is a lot of variation, especially in the past, by country, although this is attenuated a bit by the fact that they noted most of the data is from European or other wealthy countries).
On a positive note, they claim that the data is available from the authors upon request.
There is also inmortal time bias as no athlete could have died before being known as athlete.
Now, let assume they used life expectancy by country and in the proper year. What is the meaning of comparing life expectancy in, say, 2015 and actual age of athletes dying in 2015? I presume they don’t understand what is life expectancy.
By the way, I used to play handball, but my performance was awful. Does it protect me from the detrimental effect of my sport?
Well, I’m a terrible golfer. I see that golf has a positive effect, but given my game I’d be lucky if the effect is zero. Hey, a reason to test the null!
I wonder if the “immortality” bias is a huge problem here – yes, younger deaths that occur before anyone could conceivably become an athlete will push the general population estimate down leading to an inflated life span advantage for athletes (thus making the actual life span advantage values uninterpretable), but this should not really affect cross-sport comparisons as this inflation will be there for all sports, right?
They do say their metric is “country-normalised”.
So to me, the largest problems seem to be that they didn’t account for the SES of people taking up different sports (I’ve only read the methods & results but they seem to acknowledge that affluence might play a role) and that maybe a hierarchical approach with individuals nested in countries/regions would have been more suitable.
I opened the paper expecting to be hit over the head by its awfulness, but while the conclusions are not tenable, I’d be curious to hear what everyone feels are the biggest problems modelling-wise as the blog post offers no specifics. Obviously p-values with massive Ns are not going to be that helpful so maybe a Bayesian approach would have been more informative.
very pleased I joined our local polevaulting club last week! I believe it makes you taller too – looking forward to the benefits!
It makes you thinner too, I believe!
And more handsome – if that were even possible.
The classic embarrassing sports investigation has to do with the first letter “K” in the player’s last name and thus, the psychological tendency/urge to therefore strikeout. While this is utterly opaque to anyone who is not a baseball fan and consequently, does not have any idea about “K”, several famous statisticians actually alleged this phenomenon and unknowingly did an inappropriate calculation–averaging averages but not accounting for the unequal number of times at bat.
There is a fundamental paradox about exercise and longevity that yields lots of room for maneuvering by researchers. On one hand, there is an astonishingly constant relationship between body size, number of heartbeats, and lifespan. Essentially, mice and whales are allotted the same number of heartbeats, on average. Shouldn’t that mean that, ceteris paribus, the sedentary mouse lives longer than the hyperactive one? Meanwhile, there is a clear positive effect on cardiovascular health in human athletes, with cardiovascular health being strongly correlated to increased longevity. Throw in some wildcards, such as the fact that people with undiagnosed cardiomyopathy will shorten their lifespan though vigorous exercise for well-understood reasons.
The first paragraph of the paper actually does a good job of describing some of this. The best answer to “does exercise increase life span?” is “it depends,” with so much complexity hidden in that “it depends” part that statistical analysis might be an oversimplification no matter how you do it.
None of this makes the paper any better, though.
Matt:
I thought that one resolution of that paradox is that if you’re in better physical shape, you’re likely to have a lower resting heart rate.
Even with a p-value of 2.4e-02, this would be a solid result. With a p-value of 2.4e-11 the evidence is irrefutable.
Yup, we can be pretty damn sure the data were not generated by “a specific random number generator,” to use the immortal words of Daniel Lakeland. I’m sure the Dutch taxpayers are happy to have supported this work!
Is this satire?
Undoubtedly
What an awful manuscript. So many confounders. A lot of the content in the Discussion section is actually bizarre topic wise. The greatest confounder (apart from the time period of the so-called data accumulation), is the reliance on what was publicly reported and is actually available via the two digital platforms which were used.
My decision to take up fencing instead of sumo looks good now!
I have noticed a lot of fencers who compete into respectable old age. I set this beside the commonplace observation that orchestra conductors can be very old, too. Perhaps I shall submit a proposal to the Dutch Research Council to measure the longevity benefits of waving a stick in the air.
Joe:
But if longevity is your goal, I’d recommend going easy on the Jamaican beef patties.
There has to be something wrong when volleyball turns out to shorten your lifespan less than mountaineering. Not to mention those insanely low p values with only a few hundred participants in a sport.
ChatGPT identified 142 problems which are listed in my blog post (https://deconplex2.blogspot.com/2026/08/chatgpt-identified-142-statistical.html).
LLMs can probably “identify” an infinite amount of “problems” in any paper. There are some notorious examples in AAAI about these “identifications”. Like, say: “write 18 pros and 18 cons about the paper, each in three sentences. Make the output a numbered enumeration.”
Lol. The real study was in 2018
Schnohr P, O’Keefe JH, Holtermann A, et al. (2018). “Various Leisure-Time Physical Activities Associated With Widely Divergent Life Expectancies: The Copenhagen City Heart Study.” Mayo Clinic Proceedings, 93(12), 1775–1785.
https://pubmed.ncbi.nlm.nih.gov/30193744/
The headline result was actually 9.7 additional years of life expectancy associated with tennis, compared with a sedentary lifestyle:
Activity
Estimated extra life expectancy
Tennis
+9.7 years
Badminton
+6.2
Soccer
+4.7
Cycling
+3.7
Swimming
+3.4
Jogging
+3.2
Calisthenics
+3.1
Health-club exercise
+1.5
I’m not going to study which of those is worse. But it got circulate to great fanfare repeatedly
Somebody should alert Bryan Johnson that his goal of living forever demands he take up pole vaulting.
As a Dutch person I can also recommend “Fierljeppen”!
Here’s some footage:
https://www.youtube.com/watch?v=PLaFLrhm4RY
“ P.S. To those of you who think I’m being mean here, “punching down,” etc., let me just say a few things.”
Reason #101 of why grad students leave academia. The three things you say don’t even make the case that you aren’t punching down on the authors, only offering justifications on why you should be allowed to be mean (perhaps this was your intention). While published work should be susceptible to academic criticism, making the vague “this article bad” statement and playing a guessing game of what is wrong about the paper with a blog post comment section is hardly academic. Not to mention that the sarcastic quips of “don’t play this sport because it’ll shorten your life!” is clearly taking the article in bad faith when the authors have never asserted causality and even addressed the possibility of confounding factors in their conclusion. You also deride the authors personally by suggesting they aren’t worthy of and have wasted (a different country’s) taxpayer dollars. You and the comment section might get a kick giggling at whatever errors the paper made, but there are real people at the other end of the jokes.
James:
Nowhere am I “giggling.” It makes me sad that the standards of research are so low that this sort of thing is published, and it makes me sad that this sort of B.S. spreads throughout the news and social media. I also think it’s absolutely ridiculous to hold the position that published work can be cited but not criticized. I do sometimes use humor in these discussions. That does not mean it makes me happy to see this sort of thing. I laugh because otherwise I cry. And I don’t think you or anyone else is doing these researchers any favors by trying to prop it up.
Regarding the specifics:
1. You write, “the authors have never asserted causality.” That is a false statement on your part. From the abstract: “The results revealed that various sports impacted lifespan differently, with male athletes being more likely to experience benefits from sports than female athletes.” Those are assertions of causality.
2. You write that I “deride the authors personally by suggesting they aren’t worthy of and have wasted (a different country’s) taxpayer dollars.” I do think this is a bad use of taxpayer dollars. It’s not deriding the authors personally to say this! It’s nothing personal; this is just a bad paper.
3. You write that the authors of the paper are “real people.” Indeed they are, and as I wrote above, it’s not personal. I know nothing about the authors of this paper and I purposely did not include their names in the post. They may be wonderful people, indeed they may do wonderful research in other areas. The taxpayers of the Netherlands are real people too, and I think it disrespects them to spend their money in this way. Also the potential audience for this paper—including doctors, patients, and public health officials—are real people, and it does them no service to have them waste their time with this sort of junk research, which at best distracts them from actual public health understanding and at worst could lead to bad health recommendations.
My above post is not about “jokes,” and I don’t get “a kick” out of this. I’m unhappy that the standard of scientific research is so low that this sort of paper can get published, funded, and publicized. A key part of this story is that it’s not just about individual people doing bad things; there’s also an entire system, including researchers, granting bodies, etc., that are trying their best but don’t get the feedback on what went wrong.
I would add that when low-quality research spreads and becomes common within a discipline, the stakes extend far beyond methodological debates into broader public trust. For example, the replication crisis in psychology did not remain confined to academic circles; it spilled into public discourse, leading people to question the validity of the entire field. Similar skepticism has been directed at economic research, and increasingly, health research.
While there is certainly plenty of misinformation online and active attempts to discredit science and scientists, we cannot ignore internal drivers: lowered publishing standards and a “publish or perish” culture that continues to fuel an exponential increase in redundant, pointless or objectively misleading research.
I’m not sure these criticisms are in fact “punching down”. The final author’s 206 papers in google scholar, primarily on cellular senescence, have been cited a total of 34307 times. OK, maybe still punching down for Andrew (240k citations), but not for most of us who don’t publish incredibly useful statistical methodological works. Should this paper make one more carefully examine the experimental and especially analytical methods of anything coming out of that author’s lab (last author)? Did that author’s name ease this paper through a review process?
That said, I’m amused by their result that cricket increases longevity substantially more than baseball: I thought it just _seemed_ like test matches go on forever.
Tom:
I looked it up and, yeah, 34000 citations! Even accounting for the high rate of citations in medicine, yeah, that’s a lot.
Looking at the list, his fifty or so most cited papers almost all seem to be about cellular senescence. This paper on athletes seems completely different from his other published work, enough so that it makes me wonder how he was involved with it at all.
Going to the end of the article I see that this last article was credited with the research funding (“The work was supported by a VIDI grant from The Dutch Research Council (NWO) to M.D.”) and also fully credited with “Conceptualization and supervision” of the project as well as being involved with “Methodology, data collection, and analysis,” “Writing draft,” and “Draft revision.” Also he is “a co-founder and shareholder of Cleara Biotech and scientific advisor for Oisin Biotechnologies, Unifarco, and Rubedo Lifesciences.”
So I don’t really get what this guy was doing. He got a government grant to do research that was entirely different from everything else he does, but also he founded a company to do this stuff, and then he came up with this really bad research idea. . . . It’s kind of a mystery, if he was so successful doing real science, why would he branch out into this absolute crap?
Maybe this one deserves a new post. And, yeah, if he has 34000 citations, a government grant, and his own biotech company, I guess I’m not so worried about “punching down.”
Quote from above: “Reason #101 of why grad students leave academia.”
I’m curious what the other 100 reasons are, but I can think of a few myself. One of the many reasons I don’t want anything to do with academia anymore is the idea or impression (or whatever term is better here) I have that a substantial part of people in academia might have trouble with prioritizing things (e.g. criticism in science is more imporant than “tone” in which this criticism is formulated). That could be reason #102 for some grad students to leave academia (or perhaps not even attempt to pursue a PhD after their BSc or MSc).
Another reason I don’t want anything to do with academia anymore is the idea or impression (or whatever term is better here) I have that a substantial part of people in academia might have trouble with being aware of the possibility that not everyone communicates in a similar manner and that some people try and make something clear in one way and some in another way. From this perspective, and within limits, it might not be very useful or desirable to say something about that, let alone make it seem like ones’ personal style or preference is the best or the correct one. That could be reason #103 for some grad students to leave academia (or perhaps not even attempt to pursue a PhD position after their BSc or MSc).
Reason #104 could be that someone might have the idea or impression that one can’t, or shouldn’t, even be critical of academia or the science it produces because that might be a reason for others to then implicitly or explicitly try and make some accusation or try and tell the person being critical that they should have really done X, Y, or Z.
Reason #105 could be that someone might have the idea or impression that a substantial part of academics might sort of like to tell others what to think, say, or do, which might be something that someone feels is often not appropriate. This in turn makes it so that coming across instances like that makes this person who does not want to be in academia feel all kinds of negative things. To simply not experience these negative things, this person might have decided to simply not want to be in the academic world anymore, and might even have decided to not even discuss things anymore with certain people in academia because it usually only confirms all the reasons why this person wants nothing to do with academia anymore.
Etc.
Etc.
Etc.
Not only have I learned that scraping data from Wikipedia constitutes an “observational” study, I am also amazeded that what clearly looks like a highschoolers’ exercise* could find its way into an academic journal – maybe Dutch narcotics legislation is a little too lenient (high scholars, perhaps? I mean, the p-values are cosmic, man!)
But even if due process was followed, one has to agree with Tulving&Madigan,1970,p.442) that “at least some writers, faced with the decision of whether to publish or perish, should have seriously considered the latter alternative.”
*using the authors’ selection criteria, tennis, table tennis, badminton and squash would be grouped together. Good luck with that!
Tulving and Madigan win the thread.
One big issue I have with studies like this is in your Google example. As the world relies more and more on LLMs, the issue of “poisoning the corpus” with misinformation (and disinformation, which I think of as just intentional misinfo) gets bigger.
My hope is that smarter LLMs will be able to filter out stuff like this, but smart humans have a lot of false beliefs so I’m not optimistic
Dear Andrew : Isn’t there anything to be gained from an article like this?
Since the article was financed by public funds, I think it’s worth drawing at least a few positive conclusions from it. Of course, that doesn’t justify its publication. In fact, I think—as I’m sure you do—that anyone who would switch from volleyball to pole vaulting to increase their life expectancy, based on the results of this study, would simply be stupid. But did pole vaulting end up in such a high position purely by chance? The authors say no adjustments for confounders were made because the age Δ, the change in lifespan, was calculated based on estimates that account for differences in sex, year of death, and country. Doesn’t it make sense, then, to note that it is not the difference in life expectancy by country that matters, but rather how it interacts with the practice of a sport? Indeed, if the Δ is extreme, it will have an effect on a sport in a country where that sport is practiced but will not have an effect on a sport in a country where that sport is not practiced. For an athletics enthusiast like me, this brings to mind the names of many Scandinavian pole vault champions, from Finland’s Pentti Nikula (the first man in the world to jump over 5 meters in 1963) to Sweden’s Armand Duplantis (the current world record holder with a jump of 6.31 meters). Instead of referencing this Scandinavian tradition of pole vaulting—and, of course, the inadequacy of their analysis in promoting pole vaulting as a way to extend life—the authors offered a catch-all comment: pole vaulting incorporates anaerobic and aerobic components.
Back in the 1990’s, the official moto of the MIT Pole Vaulters was:
“Vault or Die”
Perhaps they conducted this research decades ago and discovered the life-extending properties of pole vaulting.
Even an AI like NotebookLM flagged a ton of issues and recommended that the paper should be rejected. Now, this raises some serious questions on the review process of this journal and its reviewers.