Our article has appeared in The American Scientist. (Here’s a link to the full article; hit control-plus to make the font more readable.) I highly recommend it for your introductory (or advanced) statistics classes. We start with a silly story of a flawed statistical analysis of sex ratios that managed to sneak into a serious scientific journal, then discuss general issues of how to interpret inconclusive statistical findings (including a brief analysis of data from People Magazine’s 50 Most Beautiful People lists), and then loop back and discuss the statistical reasons that exaggerated claims can get amplified by the news media.
The article begins as follows:
In the past few years, Satoshi Kanazawa, a reader in management and research methodology at the London School of Economics, published a series of papers in the Journal of Theoretical Biology with titles such as “Big and Tall Parents Have More Sons” (2005), “Violent Men Have More Sons” (2006), “Engineers Have More Sons, Nurses Have More Daughters” (2005), and “Beautiful Parents Have More Daughters” (2007). More recently, he has publicized some of these claims in an article, “10 Politically Incorrect Truths About Human Nature,” for Psychology Today and in a book written with Alan S. Miller, Why Beautiful People Have More Daughters.
However, the statistical analysis underlying Kanazawa’s claims has been shown to have basic flaws, with some of his analyses making the error of controlling for an intermediate outcome in estimating a causal effect, and another analysis being subject to multiple-comparisons problems. These are technical errors (about which more later) that produce misleading results. In short, Kanazawa’s findings are not statistically significant, and the patterns he analyzed could well have occurred by chance. Had the lack of statistical significance been noticed in the review process, these articles would almost certainly not have been published in the journal. The fact of their appearance (and their prominence in the media and a popular book) leads to an interesting statistical question: How should we think about research findings that are intriguing but not statistically significant? . . .
We also discuss Why Is This Important? and Why Is This Not Obvious?