Skip to content
Archive of posts filed under the Sociology category.

What’s the p-value good for: I answer some questions.

Martin King writes: For a couple of decades (from about 1988 to 2006) I was employed as a support statistician, and became very interested in the p-value issue; hence my interest in your contribution to this debate. (I am not familiar with the p-value ‘reconciliation’ literature, as published after about 2005.) I would hugely appreciate […]

Automation and judgment, from the rational animal to the irrational machine

Virgil Kurkjian writes: I was recently going through some of your recent blog posts and came across Using numbers to replace judgment. I recently wrote something about legible signaling which I think helps shed some light on exactly what causes the bureaucratization of science and maybe what we can do about it. In short I […]

Are statistical nitpickers (e.g., Kaiser Fung and me) getting the way of progress or even serving the forces of evil?

As Ira Glass says, today we have a theme and some variations on this theme. Statistical nitpickers: Do they cause more harm than good? I’d like to think we cause more good than harm, but today I want to consider the counter-argument, that, even when we are correct on the technical merits, we statisticians should […]

More on that 4/20 road rage researcher: Dude could be a little less amused, a little more willing to realize he could be on the wrong track with a lot of his research.

So, back on 4/20 we linked to the post by Sam Harper and Adam Palayew shooting down a silly article, published in JAMA and publicized around the world, that claimed excess road deaths on 4/20 (“cannabis day”). I googled the authors of that silly JAMA paper and found that one of them, Dr. Donald Redelmeier, […]

“Less Wow and More How in Social Psychology”

Fritz Strack sends along this article from 2012 which has an interesting perspective. Strack’s article begins: But, he continues, things changed in 2011 with the scandals of Diederik Stapel (a career built upon fake data), Daryl Bem (joke science getting published in a real journal), and a seemingly unending series of prominent studies that failed […]

“Troubling Trends in Machine Learning Scholarship”

Garuav Sood writes: You had expressed slight frustration with some ML/CS papers that read more like advertisements than anything else. The attached paper by Zachary Lipton and Jacob Steinhardt flags four reasonable concerns in modern ML papers: Recent progress in machine learning comes despite frequent departures from these ideals. In this paper, we focus on […]

Glenn Shafer tells us about the origins of “statistical significance”

Shafer writes: It turns out that Francis Edgeworth, who introduced “significant” in statistics, and Karl Pearson, who popularized it in statistics, used it differently than we do. For Edgeworth and Pearson, “being significant” meant “signifying”. An observed difference was significant if it signified a real difference, and you needed a very small p-value to be […]

Junk science and fake news: Similarities and differences

Jingyi Kenneth Tay writes: As I read your recent post, “How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions” . . . and still stays around even after it’s been retracted, I realized that there are many similarities between this and fake news: how it is much easier to put fake news out […]

Funny citation-year thing

So, I’m going through the final draft of Regression and Other Stories, adding index entries, cleaning up references, etc., and I noticed this: Yes, I cite myself a lot—sometimes people call it “self-citation” and act like it’s a bad thing—but I think it’s helpful to point people to my earlier writings on various topics. Anyway, […]

They misreport their experiments and don’t fess up when they’ve been caught.

Javier Benitez points us to this paper, “COMPare: Qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials,” by Ben Goldacre, Henry Drysdale, Cicely Marston, Kamal Mahtani, Aaron Dale, Ioan Milosevic, Eirion Slade, Philip Hartley and Carl Heneghan, who write: Discrepancies between pre-specified and reported outcomes are an important and […]

The State of the Art

Jesse Singal writes: This was presented, in Jennifer Eberhardt’s book Biased, as evidence to support the idea that even positive portrayals of black characters could be spreading and exacerbating unconscious antiblack bias. I did not see evidence to support that idea. I replied: I don’t understand what you’re saying here. I clicked thru and the […]

“Suppose that you work in a restaurant…”

In relation to yesterday’s post on Monty Hall, Josh Miller sends along this paper coauthored with the ubiquitous Adam Sanjurjo, “A Bridge from Monty Hall to the Hot Hand: The Principle of Restricted Choice,” which begins: Suppose that you work in a restaurant where two regular customers, Ann and Bob, are equally likely to come […]

“Superior: The Return of Race Science,” by Angela Saini

“People so much wanted the story to be true . . . that they couldn’t look past it to more mundane explanations.” – Angela Saini, Superior. I happened to be reading this book around the same time as I attended the Metascience conference, which was motivated by the realization during the past decade or so […]

“Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science”

As promised, let’s continue yesterday’s discussion of Christopher Tong’s article, “Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science.” First, the title, which makes an excellent point. It can be valuable to think about measurement, comparison, and variation, even if commonly-used statistical methods can mislead. This reminds me of the idea in decision analysis […]

Harking, Sharking, Tharking

Bert Gunter writes: You may already have seen this [“Harking, Sharking, and Tharking: Making the Case for Post Hoc Analysis of Scientific Data,” John Hollenbeck, Patrick Wright]. It discusses many of the same themes that you and others have highlighted in the special American Statistician issue and elsewhere, but does so from a slightly different […]

Things I didn’t have time to talk about yesterday at the Metascience conference

– Sane, reasonable, . . . and wrong. Here I was going to talk about some reasonable and moderate-sounding recommendations that I think miss some key issues. – The fractal nature of scientific revolution. I’ve talked about this a lot, for example in 2005, 2007, and 2012. – Workflow. My plan here was to talk […]

A world of Wansinks in medical research: “So I guess what I’m trying to get at is I wonder how common it is for clinicians to rely on med students to do their data analysis for them, and how often this work then gets published”

In the context of a conversation regarding sloppy research practices, Jordan Anaya writes: It reminds me of my friends in residency. Basically, while they were med students for some reason clinicians decided to get them to analyze data in their spare time. I’m not saying my friends are stupid, but they have no stats or […]

It’s not just p=0.048 vs. p=0.052

Peter Dorman points to this post on statistical significance and p-values by Timothy Taylor, editor of the Journal of Economic Perspectives, a highly influential publication of the American Economic Association. I have some problems with what Taylor writes, but for now I’ll just take it as representing a certain view, the perspective of a thoughtful […]

The ladder of social science reasoning, 4 statements in increasing order of generality, or Why didn’t they say they were sorry when it turned out they’d messed up?

Is the effect they found too large to believe? (the effect of breakfast macronutrients on social decisions)

Someone who wishes to remain anonymous writes: Have you seen this paper? I [my correspondent] don’t see any obvious problems, but the results fall into the typical social psychology case “unbelievably large effects of small manipulations”. They even say so themselves: We provided converging evidence from two studies showing that a relatively small variation in […]