Skip to content
Archive of posts filed under the Causal Inference category.

One reason why that estimated effect of Fox News could’ve been so implausibly high.

Ethan Kaplan writes: I just happened upon a post of yours on the potential impact of Fox News on the 2016 election [“No, I don’t buy that claim that Fox news is shifting the vote by 6 percentage points“]. I am one of the authors of the first Fox News study from 2007 (DellaVigna and […]

What’s the biggest mistake revealed by this table? A puzzle:

This came up in our discussion the other day: It’s a table comparing averages for treatment and control groups in an experiment. There’s one big problem here (summarizing differences by p-values) and some little problems, such as reporting values to ridiculous precision (who cares if something has an average of “346.57” when its standard deviation […]

Adjusting for differences between treatment and control groups: “statistical significance” and “multiple testing” have nothing to do with it

Jonathan Falk points us to this post by Scott Alexander entitled “Two Unexpected Multiple Hypothesis Testing Problems.” The important questions, though, have nothing to do with multiple hypothesis testing or with hypothesis testing at all. As is often the case, certain free-floating scientific ideas get in the way of thinking about the real problem. Alexander […]

Many years ago, when he was a baby economist . . .

Jonathan Falk writes: Many years ago, when I was a baby economist, a fight broke out in my firm between two economists. There was a question as to whether a particular change in the telecommunications laws had spurred productivity improvements or not. There a trend of x% per year in productivity improvements that had gone […]

Estimating the college wealth premium: Not so easy

Dale Lehman writes: Here’s the article referenced on Marginal Revolution today. I thought it might be of interest and worth blogging about. It is quite thorough and fairly complex. The results are quite striking – and important. My big concern relates to a critical variable – financial literacy. On page 14 they claim that it […]

Postdoctoral opportunity with Sarah Cowan and Jennifer Hill: causal inference for Universal Basic Income (UBI)

See below from Sarah Cowan: I write to announce the launch of the Cash Transfer Lab. Our mission is to build an evidence base regarding cash transfer policies like a Universal Basic Income. We answer the fundamental questions of how a Universal Basic Income policy would transform American families, communities and economies. The first major […]

PhD student and postdoc positions in Norway for doing Bayesian causal inference using Stan!

Guido Biele writes: I have two positions for a postdoc and PhD student open in a project where we will use observational data from Norwegian National registries, structural models (or the potential outcomes framework, the main thing is that we want to think systematically about identification), and Bayesian estimation in Stan to estimate causal effects […]

Regression discontinuity analysis is often a disaster. So what should you do instead? Here’s my recommendation:

Summary If you have an observational study with outcome y treatment variable z and pre-treatment predictors X, and treatment assignment depends only on X, then you can estimate the average causal effect by regressing y on z and X and looking at the coefficient of z. If there is lack of complete overlap in X […]

Bayesian methods and what they offer compared to classical econometrics

A well-known economist who wishes to remain anonymous writes: Can you write about this agent? He’s getting exponentially big on Twitter. The link is to an econometrician, Jeffrey Wooldridge, who writes: Many useful procedures—shrinkage, for example—can be derived from a Bayesian perspective. But those estimators can be studied from a frequentist perspective, and no strong […]

This one pushes all my buttons

August Wartin writes: Just wanted to make you aware of this ongoing discussion about an article in JPE: It’s the same professor Lidbom that was involved in this discussion a few years ago (I believe you mentioned something about it on your blog). Indeed, we blogged it here. Here’s the abstract of Lidbom’s more recent […]

Statistical fallacies as they arise in political science (from Bob Jervis)

Bob Jervis sends along this fun document he gives to the students in his classes. Enjoy. Theories of International Relations Assume that all the facts and assertions in these paragraphs are correct. Why do the conclusions not follow? (This does not mean that the conclusions are actually false.) What are the alternative explanations for the […]

Drew Bailey on backward causal questions and forward causal inference

Following up on my paper with Guido on backward causal questions and forward causal inference, education researcher Drew Bailey writes: (1) Some disagreements between social scientists or between social scientists and the public arise when one side is in “forward causal inference” mode and the other side is in “backward causal question” mode; (2) Individuals […]

Simulated-data experimentation: Why does it work so well?

Someone sent me a long question about a complicated social science problem involving intermediate outcomes, errors in predictors, latent class analysis, path analysis, and unobserved confounders. I got the gist of the question but it didn’t quite seem worth chasing down all the details involving certain conclusions to be made if certain affects disappeared in […]

The “story time” is to lull us in with a randomized controlled experiment and as we fall asleep, feed us less reliable conclusions that come from an embedded observational study.

Kaiser Fung explains. This comes up a lot, and his formulation in the above title is a good way of putting it. He also has this discussion of the AstraZeneca-Oxford vaccine trial results which makes me want to just do a damn Bayesian analysis of it already. I’ll have to find someone with the right […]

One more cartoon like this, and this blog will be obsolete.

This post is by Phil. This SMBC cartoon seems to wrap up about half of the content of this blog.  Of course I’m exaggerating. There will still be room for book reviews and cat photos.

Will the pandemic cause a decline in births? We’ll be able to resolve this particular debate in about 9 months . . .

The fallacy of the one-sided bet I’m gonna be talking about a news article and research paper asking the question, “Will coronavirus cause a baby boom, or is that just a myth?” And my problem is the fallacy of the one-sided bet: By asking the question, is there a positive effect or is it zero, […]

What about that new paper estimating the effects of lockdowns etc?

A couple people pointed me to this article, “Assessing Mandatory Stay‐at‐Home and Business Closure Effects on the Spread of COVID‐19,” which reports: The most restrictive non‐pharmaceutical interventions (NPIs) for controlling the spread of COVID‐19 are mandatory stay‐at‐home and business closures. . . . We evaluate the effects on epidemic case growth of more restrictive NPIs […]

“They adjusted for three hundred confounders.”

Alexey Guzey points to this post by Scott Alexander and this research article by Elisabetta Patorno, Robert Glynn, Raisa Levin, Moa Lee, and Krista Huybrechts, and writes: I [Guzey] am extremely skeptical of anything that relies on adjusting for confounders and have no idea what to think about this. My intuition would be that because […]

Does regression discontinuity (or, more generally, causal identification + statistical significance) make you gullible?

Yes basically. This one’s pretty much a perfect example of overfitting, finding a discontinuity out of noise, in that if you just draw a smooth line through each graph, it actually looks better than the discontinuous version. We see this a lot: There’s no discontinuity in the data, but it’s possible to make a discontinuity […]

No, this senatorial stock-picking study does not address concerns about insider trading:

Jonathan Falk writes: As you have tirelessly promoted, a huge problem with NHST is that “insignificant” effects on average can mask, via attenuation bias, important changes in subgroups. Further, as you have somewhat less tirelessly pointed out, you need much bigger samples to reliably see anything in subgroups, particularly when (ok.. you’re back to your […]

Where can you find the best CBD products? CBD gummies made with vegan ingredients and CBD oils that are lab tested and 100% organic? Click here.