Skip to content
Archive of posts filed under the Sociology category.

In research as in negotiation: Be willing to walk away, don’t paint yourself into a corner, leave no hostages to fortune

There’s a saying in negotiation that the most powerful asset is the ability to walk away from the deal. Similarly, in science (or engineering, business decision making, etc.), you have to be willing to give up your favorite ideas. When I look at various embarrassing examples in science during the past decade, a common thread […]

I (inadvertently) misrepresented others’ research in a way that made my story sound better.

During a recent talk (I think it was this one on statistical visualization), I spent a few minutes discussing a political science experiment involving social stimuli and attitudes toward redistribution. I characterized the study as being problematic for various reasons (for background, see this post), and I remarked that you shouldn’t expect to learn much […]

Is “abandon statistical significance” like organically fed, free-range chicken?

The question: is good statistics scalable? This comes up a lot in discussions on abandoning statistical significance, null-hypothesis significance testing, p-value thresholding, etc. I recommend accepting uncertainty, but what if it’s decision time—what to do? How can the world function if the millions of scientific decisions currently made using statistical significance somehow have to be […]

Instead of replicating studies with problems, let’s replicate the good studies. (Consider replication as an honor, not an attack.)

Commenter Thanatos Savehn pointed to an official National Academy of Sciences report on Reproducibility and Replicability that included the following “set of criteria to help determine when testing replicability may be warranted”: 1) The scientific results are important for individual decision-making or for policy decisions. 2) The results have the potential to make a large […]

Hey! Participants in survey experiments aren’t paying attention.

Gaurav Sood writes: Do survey respondents account for the hypothesis that they think people fielding the survey have when they respond? The answer, according to Mummolo and Peterson, is not much. Their paper also very likely provides the reason why—people don’t pay much attention. Figure 3 provides data on manipulation checks—the proportion guessing the hypothesis […]

“The paper has been blind peer-reviewed and published in a highly reputable journal, which is the gold standard in scientific corroboration. Thus, all protocol was followed to the letter and the work is officially supported.”

Robert MacDonald points us to this news article by Esther Addley: It’s another example of what’s probably bad science being published in a major journal, where other researchers point out its major flaws and the author doubles down. In this case, the University of Bristol has an interesting reaction. It’s pulled down its article praising […]

The climate economics echo chamber: Gremlins and the people (including a Nobel prize winner) who support them

Jay Coggins, a professor of applied economics at the university of Minnesota, writes in with some thoughts about serious problems of within the field of environmental economics: Your latest on Tol [a discussion of a really bad paper he published in The Review of Environmental Economics and Policy, “the official journal of the Association of […]

Kool-aid != Dogfood, and Certainty is no substitute for knowledge.

Palko quotes from this news article by Shirin Ghaffary quoting Marcelo Claure, the new executive chairman of the recent Ponzi scheme WeWork. Here’s Claure, in a speech to WeWork employees: [W]e got to drink our own Kool-aid, we got to make sure that if we’re selling this magic to others, we got to have this […]

“Any research object with a strong and obvious series of inconsistencies may be deemed too inaccurate to trust, irrespective of their source. In other words, the description of inconsistency makes no presumption about the source of that inconsistency.”

Nick Brown and James Heathers write: We have seen two documents from the Scientific Integrity Officer at the University of Rennes-2 . . . The first of these dates from June 2018 and is entitled (our translation from French), “Preliminary Investigation Report Regarding the Allegations of Fraud against Nicolas Guéguen”. . . . We would […]

“Bullshitters. Who Are They and What Do We Know about Their Lives?”

Hannes Margraf writes: I write to make you aware of a paper with the delightful title “Bullshitters. Who Are They and What Do We Know about Their Lives?” [by John Jerrim, Phil Parker, and Nikki Shure]. The authors examine “teenagers’ propensity to claim expertise in three mathematics constructs that do not really exist” and “find […]

“The Role of Nature versus Nurture in Wealth and Other Economic Outcomes and Behaviors”

Sandra Black, Paul, Devereux, Petter Lundborg, and Kaveh Majlesi write: Wealth is highly correlated between parents and their children; however, little is known about the extent to which these relationships are genetic or determined by environmental factors. We use administrative data on the net wealth of a large sample of Swedish adoptees merged with similar […]

The real lesson learned from those academic hoaxes: a key part of getting a paper published in a scholarly journal is to be able to follow the conventions of the journal. And some people happen to be good at that, irrespective of the content of the papers being submitted.

I wrote this email to a colleague: Someone pointed me to this paper. It’s really bad. It was published by The Review of Environmental Economics and Policy, “the official journal of the Association of Environmental and Resource Economists and the European Association of Environmental and Resource Economists.” Is this a real organization? The whole thing […]

“Here’s an interesting story right in your sweet spot”

Jonathan Falk writes: Here’s an interesting story right in your sweet spot: Large effects from something whose possible effects couldn’t be that large? Check. Finding something in a sample of 1024 people that requires 34,000 to gain adequate power? Check. Misuse of p values? Check Science journalist hype? Check Searching for the cause of an […]

The status-reversal heuristic

Awhile ago we came up with the time-reversal heuristic, which was a reaction to the common situation that there’s a noisy study, followed by an unsuccessful replication, but all sorts of people want to take the original claim as the baseline and construct high walls to make it difficult to move away from that claim. […]

How to think scientifically about scientists’ proposals for fixing science

I kinda like this little article which I wrote a couple years ago while on the train from the airport. It will appear in the journal Socius. Here’s how it begins: Science is in crisis. Any doubt about this status has surely been been dispelled by the loud assurances to the contrary by various authority […]

Rachel Tanur Memorial Prize for Visual Sociology

Judith Tanur writes: The Rachel Tanur Memorial Prize for Visual Sociology recognizes students in the social sciences who incorporate visual analysis in their work. The contest is open worldwide to undergraduate and graduate students (majoring in any social science). It is named for Rachel Dorothy Tanur (1958–2002), an urban planner and lawyer who cared deeply […]

When presenting a new method, talk about its failure modes.

A coauthor writes: I really like the paper [we are writing] as it is. My only criticism of it perhaps would be that we present this great new method and discuss all of its merits, but we do not really discuss when it fails / what its downsides are. Are there any cases where the […]

On the term “self-appointed” . . .

I was reflecting on what bugs me so much about people using the term “self-appointed” (for example, when disparaging “self-appointed data police” or “self-appointed chess historians“). The obvious question when someone talks about “self-appointed” whatever is, Who self-appointed you to decide who is illegitimately self-appointed? But my larger concern is with the idea that being […]

What’s the p-value good for: I answer some questions.

Martin King writes: For a couple of decades (from about 1988 to 2006) I was employed as a support statistician, and became very interested in the p-value issue; hence my interest in your contribution to this debate. (I am not familiar with the p-value ‘reconciliation’ literature, as published after about 2005.) I would hugely appreciate […]

Automation and judgment, from the rational animal to the irrational machine

Virgil Kurkjian writes: I was recently going through some of your recent blog posts and came across Using numbers to replace judgment. I recently wrote something about legible signaling which I think helps shed some light on exactly what causes the bureaucratization of science and maybe what we can do about it. In short I […]