Skip to content
Archive of posts filed under the Causal Inference category.

Simulated-data experimentation: Why does it work so well?

Someone sent me a long question about a complicated social science problem involving intermediate outcomes, errors in predictors, latent class analysis, path analysis, and unobserved confounders. I got the gist of the question but it didn’t quite seem worth chasing down all the details involving certain conclusions to be made if certain affects disappeared in […]

The “story time” is to lull us in with a randomized controlled experiment and as we fall asleep, feed us less reliable conclusions that come from an embedded observational study.

Kaiser Fung explains. This comes up a lot, and his formulation in the above title is a good way of putting it. He also has this discussion of the AstraZeneca-Oxford vaccine trial results which makes me want to just do a damn Bayesian analysis of it already. I’ll have to find someone with the right […]

One more cartoon like this, and this blog will be obsolete.

This post is by Phil. This SMBC cartoon seems to wrap up about half of the content of this blog.  Of course I’m exaggerating. There will still be room for book reviews and cat photos.

Will the pandemic cause a decline in births? We’ll be able to resolve this particular debate in about 9 months . . .

The fallacy of the one-sided bet I’m gonna be talking about a news article and research paper asking the question, “Will coronavirus cause a baby boom, or is that just a myth?” And my problem is the fallacy of the one-sided bet: By asking the question, is there a positive effect or is it zero, […]

What about that new paper estimating the effects of lockdowns etc?

A couple people pointed me to this article, “Assessing Mandatory Stay‐at‐Home and Business Closure Effects on the Spread of COVID‐19,” which reports: The most restrictive non‐pharmaceutical interventions (NPIs) for controlling the spread of COVID‐19 are mandatory stay‐at‐home and business closures. . . . We evaluate the effects on epidemic case growth of more restrictive NPIs […]

“They adjusted for three hundred confounders.”

Alexey Guzey points to this post by Scott Alexander and this research article by Elisabetta Patorno, Robert Glynn, Raisa Levin, Moa Lee, and Krista Huybrechts, and writes: I [Guzey] am extremely skeptical of anything that relies on adjusting for confounders and have no idea what to think about this. My intuition would be that because […]

Does regression discontinuity (or, more generally, causal identification + statistical significance) make you gullible?

Yes basically. This one’s pretty much a perfect example of overfitting, finding a discontinuity out of noise, in that if you just draw a smooth line through each graph, it actually looks better than the discontinuous version. We see this a lot: There’s no discontinuity in the data, but it’s possible to make a discontinuity […]

No, this senatorial stock-picking study does not address concerns about insider trading:

Jonathan Falk writes: As you have tirelessly promoted, a huge problem with NHST is that “insignificant” effects on average can mask, via attenuation bias, important changes in subgroups. Further, as you have somewhat less tirelessly pointed out, you need much bigger samples to reliably see anything in subgroups, particularly when (ok.. you’re back to your […]

Flaxman et al. respond to criticisms of their estimates of effects of anti-coronavirus policies

As youall know, as the coronavirus has taken its path through the world, epidemiologists and social scientists have tracked rates of exposure and mortality, studied the statistical properties of the transmission of the virus, and estimated effects of behaviors and policies that have been tried to limit the spread of the disease. All this is […]

More on the Heckman curve

David Rea writes: A slightly more refined version of our paper on the Heckman Curve [discussed on blog last year] has been published in the Journal of Economic Surveys. The journal will also publish a response by James Heckman, as well as a reply from us. As you predicted, James Heckman’s critique of our work […]

“Inferring the effectiveness of government interventions against COVID-19”

John Salvatier points us to this article by Jan Brauner et al. that states: We gathered chronological data on the implementation of NPIs [non-pharmaceutical interventions, i.e. policy or behavioral interventions] for several European, and other, countries between January and the end of May 2020. We estimate the effectiveness of NPIs, ranging from limiting gathering sizes, […]

Debate involving a bad analysis of GRE scores

This is one of these academic ping-pong stories of a general opinion, an article that challenges the general opinion, a rebuttal to that article, a rebuttal to the rebuttal, etc. I’ll label the positions as A1, B1, A2, B2, and so forth: A1: The starting point is that Ph.D. programs in the United States typically […]

Covid crowdsourcing

Macartan Humphries writes: We put together a platform that lets researchers contribute predictive models of cross national (and within country) Covid mortality, focusing on political and social accounts. The plan then is to aggregate using a stacking approach. Go take a look.

Are we constantly chasing after these population-level effects of these non-pharmaceutical interventions that are hard to isolate when there are many good reasons to believe in their efficacy in the first instance?

A couple days ago we discussed issues of communicating uncertainty in a coronavirus mask experiment. That study itself is not so important, but I remain interested in the larger issues of inference and communication. I sent the discussion to epidemiologist Jon Zelner, who wrote: The struggle is real! I think this is a nice example […]

Discussion of uncertainties in the coronavirus mask study leads us to think about some issues . . .

1. Communicating of uncertainty A member of the C19 Discussion List, which is a group of frontline doctors fighting Covid-19, asked me what I thought of this opinion article, “Covid-19: controversial trial may actually show that masks protect the wearer,” published last month by James Brophy in the British Medical Journal. Brophy writes: Paradoxically, the […]

The rise and fall and rise of randomized controlled trials (RCTs) in international development

Gil Eyal sends along this fascinating paper coauthored with Luciana de Souza Leão, “The rise of randomized controlled trials (RCTs) in international development in historical perspective.” Here’s the story: Although the buzz around RCT evaluations dates from the 2000s, we show that what we are witnessing now is a second wave of RCTs, while a […]

No, I don’t believe etc etc., even though they did a bunch of robustness checks.

Dale Lehman writes: You may have noticed this article mentioned on Marginal Revolution, https://www.sciencedirect.com/science/article/abs/pii/S0167629619301237. I [Lehman] don’t have access to the published piece, but here’s a working paper version. It might be worth your taking a look. It has all the usual culprits: forking paths, statistical significance as the filter, etc etc. As usual, it […]

How the election might have looked in a world without polls

On the radio this morning it was all about how Biden’s in the lead but Trump outperformed the polls just about everywhere. What if there had been no trial-heat polls? Then maybe the reporting would be how Biden outperformed Clinton almost everywhere, but given all the problems with the economy it’s surprising Trump kept it […]

“Fake Facts in Covid-19 Science: Kentucky vs. Tennessee.”

I’m writing this on 24 Apr 2020. I’ve been posting coronavirus items immediately and pushing previously scheduled material to the end of the queue (currently Oct and Nov). But this one is already forgotten so I might as well put it in lag. When it appears, you can read it and put yourself in the […]

Piranhas in the rain: Why instrumental variables are not as clean as you might have thought

Woke up in my clothes again this morning I don’t know exactly where I am And I should heed my doctor’s warning He does the best with me he can He claims I suffer from delusion But I’m so confident I’m sane It can’t be a statistical illusion So how can you explain Piranhas in […]

Where can you find the best CBD products? CBD gummies made with vegan ingredients and CBD oils that are lab tested and 100% organic? Click here.