Skip to content
Archive of posts filed under the Causal Inference category.

Discussion of uncertainties in the coronavirus mask study leads us to think about some issues . . .

1. Communicating of uncertainty A member of the C19 Discussion List, which is a group of frontline doctors fighting Covid-19, asked me what I thought of this opinion article, “Covid-19: controversial trial may actually show that masks protect the wearer,” published last month by James Brophy in the British Medical Journal. Brophy writes: Paradoxically, the […]

The rise and fall and rise of randomized controlled trials (RCTs) in international development

Gil Eyal sends along this fascinating paper coauthored with Luciana de Souza Leão, “The rise of randomized controlled trials (RCTs) in international development in historical perspective.” Here’s the story: Although the buzz around RCT evaluations dates from the 2000s, we show that what we are witnessing now is a second wave of RCTs, while a […]

No, I don’t believe etc etc., even though they did a bunch of robustness checks.

Dale Lehman writes: You may have noticed this article mentioned on Marginal Revolution, https://www.sciencedirect.com/science/article/abs/pii/S0167629619301237. I [Lehman] don’t have access to the published piece, but here’s a working paper version. It might be worth your taking a look. It has all the usual culprits: forking paths, statistical significance as the filter, etc etc. As usual, it […]

How the election might have looked in a world without polls

On the radio this morning it was all about how Biden’s in the lead but Trump outperformed the polls just about everywhere. What if there had been no trial-heat polls? Then maybe the reporting would be how Biden outperformed Clinton almost everywhere, but given all the problems with the economy it’s surprising Trump kept it […]

“Fake Facts in Covid-19 Science: Kentucky vs. Tennessee.”

I’m writing this on 24 Apr 2020. I’ve been posting coronavirus items immediately and pushing previously scheduled material to the end of the queue (currently Oct and Nov). But this one is already forgotten so I might as well put it in lag. When it appears, you can read it and put yourself in the […]

Piranhas in the rain: Why instrumental variables are not as clean as you might have thought

Woke up in my clothes again this morning I don’t know exactly where I am And I should heed my doctor’s warning He does the best with me he can He claims I suffer from delusion But I’m so confident I’m sane It can’t be a statistical illusion So how can you explain Piranhas in […]

Reference for the claim that you need 16 times as much data to estimate interactions as to estimate main effects

Ian Shrier writes: I read your post on the power of interactions a long time ago and couldn’t remember where I saw it. I just came across it again by chance. Have you ever published this in a journal? The concept comes up often enough and some readers who don’t have methodology expertise feel more […]

Some wrong lessons people will learn from the president’s illness, hospitalization, and expected recovery

Jonathan Falk writes about the president’s illness: I [Falk] would think it provides a focused opportunity to make a few salient statistical education points. First, a 6 percent mortality rate (among old people with comorbidities) is really bad, but any single selected person is really quite unlikely to die, or even be really sick. Same […]

Randomized but unblinded experiment on vitamin D as a coronavirus treatment. Let’s talk about what comes next. (Hint: it involves multilevel models.)

Under the heading, “Here we go again,” Dale Lehman writes: If you want to blog on the continuing theme – try this (it’s from Marginal Revolution, the citation): https://marginalrevolution.com/marginalrevolution/2020/09/a-vitamin-d-bet.html https://www.sciencedirect.com/science/article/pii/S0960076020302764 Vitamin D Can Likely End the COVID-19 Pandemic What is striking is the analysis by the Rootclaim group – repeated reliance on p values as […]

A question of experimental design (more precisely, design of data collection)

An economist colleague writes in with a question: What is your instinct on the following. Consider at each time t, 1999 through 2019, there is a probability P_t for some event (e.g., it rains on a given day that year). Assume that P_t = P_1999 + (t-1999)A. So P_t has a linear time trend. What […]

Update on social science debate about measurement of discrimination

Dean Knox writes: Following up on our earlier conversation, we write to share a new, detailed examination of the article, Deconstructing Claims of Post-Treatment Bias in Observational Studies of Discrimination, by Johann Gaebler, William Cai, Guillaume Basse, Ravi Shroff, Sharad Goel, and Jennifer Hill (GCBSGH). Here’s our new paper, Using Data Contaminated by Post-Treatment Selection?, […]

Facemasks in Germany

August Torngren Wartin pointed us to this article, “Unmasked! The effect of face masks on the spread of COVID-19,” by Timo Mitze, Reinhold Kosfeld, Johannes Rode, and Klaus Wälde, and asked what I thought. My reply: I’ve not looked at it in detail but it seems reasonable. I’m sharing this for a few reasons. First, […]

This is your chance to comment on the U.S. government’s review of evidence on the effectiveness of home visiting. Comments are due by 1 Sept.

Emily Sama-Miller writes: The federally sponsored Home Visiting Evidence of Effectiveness (HomVEE) systematic evidence review is seeking public comment on proposed updates to its standards and procedures. HomVEE reviews research literature on home visiting for families with pregnant women and children from birth to kindergarten entry, and its results are used to inform federal funding […]

Somethings do not seem to spread easily – the role of simulation in statistical practice and perhaps theory.

Unlike Covid19, somethings don’t seem to spread easily and the role of simulation in statistical practice (and perhaps theory) may well be one of those. In a recent comment, Andrew provided a link to an interview about the new book Regression and Other Stories by Aki Vehtari, Andrew Gelman, and Jennifer Hill. An interview that covered […]

“100 Stories of Causal Inference”: My talk tomorrow at the Online Causal Inference Seminar

Tues 4 Aug, 11:30am on zoom: 100 Stories of Causal Inference In social science we learn from stories. The best stories are anomalous and immutable. We shall briefly discuss the theory of stories, the paradoxical nature of how we learn from them, and how this relates to forward and reverse causal inference. Then we will […]

“The Taboo Against Explicit Causal Inference in Nonexperimental Psychology”

Kevin Lewis points us to this article by Michael Grosz, Julia Rohrer, and Felix Thoemmes, who write: Causal inference is a central goal of research. However, most psychologists refrain from explicitly addressing causal research questions and avoid drawing causal inference on the basis of nonexperimental evidence. We argue that this taboo against causal inference in […]

BMJ update: authors reply to our concerns (but I’m not persuaded)

Last week we discussed an article in the British Medical Journal that seemed seriously flawed to me, based on evidence such as the above graph. At the suggestion of Elizabeth Loder, I submitted a comment to the paper on the BMJ website. Here’s what I wrote: I am concerned that the model does not fit […]

The importance of descriptive social science and its relation to causal inference and substantive theories

Here’s the abstract to a recent paper, Escaping Malthus: Economic Growth and Fertility Change in the Developing World, by Shoumitro Chatterjee and Tom Vogl: Following mid-twentieth century predictions of Malthusian catastrophe, fertility in the developing world more than halved, while living standards more than doubled. We analyze how fertility change related to economic growth during […]

Would we be better off if randomized clinical trials had never been born?

This came up in discussion the other day. In statistics and medicine, we’re generally told to rely when possible on the statistically significance (or lack of statistical significance) of results from randomized trials. But, as we know, statistical significance has all sorts of problems, most notably that it ignores questions of cost and benefit, and […]

Please socially distance me from this regression model!

A biostatistician writes: The BMJ just published a paper using regression discontinuity to estimate the effect of social distancing. But they have terrible models. As I am from Canada, I had particular interest in the model for Canada, which is on their supplemental material, page 84 [reproduced above]. I could not believe this was published. […]