High-profile statistical errors occur in the physical sciences too, it’s not just a problem in social science.

In an email with subject line, “Article full of forking paths,” John Williams writes:

I thought you might be interested in this article by John Sabo et al., which was the cover article for the Dec. 8 issue of Science. The article is dumb in various ways, some of which are described in the technical comment on it that I have submitted, but it also exhibits multiple forking paths, a lack of theory, and abundant jargon. It is also very carelessly written and reviewed. For example, the study analyzed the Mekong River stage (level of the water with respect to a reference point), but refers more often to the discharge (volume per time past a reference point: the relationship between the two is non-linear). It is pretty amazing that something like this got published.

I shared this with a colleague who is knowledgeable about this general area of research, and my colleague wrote that he agreed with Williams’s criticisms.

Williams followed up with a new document listing forking paths in the Sabo analysis.

There’s more to the story but I’ll stop here. I just wanted to share all this because it’s good to be reminded that high-profile statistical errors occur in the physical sciences too, it’s not just a problem in social science.

8 thoughts on “High-profile statistical errors occur in the physical sciences too, it’s not just a problem in social science.

  1. Agreed. I came to psychology from biology. The stats are equally bad in biology, but with one major difference. Psychology is actively working to improve things, and when biology does begin to consider QRPs, etc. it will largely be spill over from the efforts of social scientists

Leave a Reply

Your email address will not be published. Required fields are marked *