Skip to content
Archive of posts filed under the Causal Inference category.

“Did Jon Stewart elect Donald Trump?”

I wrote this post a couple weeks ago and scheduled it for October, but then I learned from a reporter that the research article under discussion was retracted, so it seemed to make sense to post this right away while it was still newsworthy. My original post is below, followed by a post script regarding […]

Did blind orchestra auditions really benefit women?

You’re blind! And you can’t see You need to wear some glasses Like D.M.C. Someone pointed me to this post, “Orchestrating false beliefs about gender discrimination,” by Jonatan Pallesen criticizing a famous paper from 2000, “Orchestrating Impartiality: The Impact of ‘Blind’ Auditions on Female Musicians,” by Claudia Goldin and Cecilia Rouse. We’ve all heard the […]

Difference-in-difference estimators are a special case of lagged regression

Fan Li and Peng Ding write: Difference-in-differences is a widely-used evaluation strategy that draws causal inference from observational panel data. Its causal identification relies on the assumption of parallel trend, which is scale dependent and may be questionable in some applications. A common alternative method is a regression model that adjusts for the lagged dependent […]

Do regression structures affect research capital? The case of pronoun drop. (also an opportunity to quote Bertrand Russell: This is one of those views which are so absurd that only very learned men could possibly adopt them.)

A linguist pointed me with incredulity to this article by Horst Feldmann, “Do Linguistic Structures Affect Human Capital? The Case of Pronoun Drop,” which begins: This paper empirically studies the human capital effects of grammatical rules that permit speakers to drop a personal pronoun when used as a subject of a sentence. By de‐emphasizing the […]

13 Reasons not to trust that claim that 13 Reasons Why increased youth suicide rates

A journalist writes: My eye was caught by this very popular story that broke yesterday — about a study that purported to find a 30 percent (!) increase in suicides, in kids 10-17, in the MONTH after a controversial show about suicide aired. And that increase apparently persisted for the rest of the year. It’s […]

A debate about effect-size variation in psychology: Simmons and Simonsohn; McShane, Böckenholt, and Hansen; Judd and Kenny; and Stanley and Doucouliagos

A couple weeks ago, Uri Simonsohn and Joe Simmons sent me and others a note that they were writing a blog post citing some of our work and asking for us to point out anything that we find “inaccurate, unfair, snarky, misleading, or in want of a change for any reason.” I took a quick […]

Continuing discussion of status threat and presidential elections, with discussion of challenge of causal inference from survey data

Last year we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz. A couple months later we updated with Mutz’s response to Morgan’s critique. Finally, Morgan has published a reply to Mutz’s response to Morgan’s comments on Mutz’s paper. Here’s a passage that is of methodological interest: […]

“How many years do we lose to the air we breathe?” Or not.

From this Washington Post article: But . . . wait a second. The University of Chicago’s Energy Policy Institute . . . what exactly is that? Let’s do a google, then we get to the relevant page. I’m concerned because this is the group that did this report, which featured this memorable graph: See this […]

Automatic voter registration impact on state voter registration

Sean McElwee points us to this study by Kevin Morris and Peter Dunphy, who write: Automatic voter registration or AVR . . . features two seemingly small but transformative changes to how people register to vote: 1. Citizens who interact with government agencies like the Department of Motor Vehicles are registered to vote, unless they […]

Conditioning on post-treatment variables when you expect self-selection

Sadish Dhakal writes: I am struggling with the problem of conditioning on post-treatment variables. I was hoping you could provide some guidance. Note that I have repeated cross sections, NOT panel data. Here is the problem simplified: There are two programs. A policy introduced some changes in one of the programs, which I call the […]

“Incentives to Learn”: How to interpret this estimate of a varying treatment effect?

Germán Jeremias Reyes writes: I am currently taking a course on Applied Econometrics and would like to ask you about how you would interpret a particular piece of evidence. Some background: In 2009, Michael Kremer et al. published an article called “Incentives to learn.” This is from the abstract (emphasis is mine): We study a […]

Wanted: Statistical success stories

Bill Harris writes: Sometime when you get a free moment, it might be great to publish a post that links to good, current exemplars of analyses. There’s a current discussion about RCTs on a program evaluation mailing list I monitor. I posted links to your power=0.06 post and your Type S and Type M post, […]

Active learning and decision making with varying treatment effects!

In a new paper, Iiris Sundin, Peter Schulam, Eero Siivola, Aki Vehtari, Suchi Saria, and Samuel Kaski write: Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action a to take for a target […]

What sort of identification do you get from panel data if effects are long-term? Air pollution and cognition example.

Don MacLeod writes: Perhaps you know this study which is being taken at face value in all the secondary reports: “Air pollution causes ‘huge’ reduction in intelligence, study reveals.” It’s surely alarming, but the reported effect of air pollution seems implausibly large, so it’s hard to be convinced of it by a correlational study alone, […]

“Heckman curve” update: The data don’t seem to support the claim that human capital investments are most effective when targeted at younger ages.

David Rea and Tony Burton write: The Heckman Curve describes the rate of return to public investments in human capital for the disadvantaged as rapidly diminishing with age. Investments early in the life course are characterised as providing significantly higher rates of return compared to investments targeted at young people and adults. This paper uses […]

Treatment interactions can be hard to estimate from data.

Brendan Nyhan writes: Per #3 here, just want to make sure you saw the Coppock Leeper Mullinix paper indicating treatment effect heterogeneity is rare. My reply: I guess it depends on what is being studied. In the world of evolutionary psychology etc., interactions are typically claimed to be larger than main effects (for example, that […]

“The Long-Run Effects of America’s First Paid Maternity Leave Policy”: I need that trail of breadcrumbs.

Tyler Cowen links to a research article by Brenden Timpe, “The Long-Run Effects of America’s First Paid Maternity Leave Policy,” that begins as follows: This paper provides the first evidence of the effect of a U.S. paid maternity leave policy on the long-run outcomes of children. I exploit variation in access to paid leave that […]

How to approach a social science research problem when you have data and a couple different ways you could proceed?

tl;dr: Someone asks me a question, I can’t really tell what he’s talking about, so I offer some generic advice. Joe Hoover writes: An issue has come up in my subsequent analyses, which uses my MrsP estimates to explore the relationship between county-level moral values and the county-level distribution of hate groups, as defined by […]

Postdoc in Chicago on statistical methods for evidence-based policy

Beth Tipton writes: The Institute for Policy Research and the Department of Statistics is seeking applicants for a Postdoctoral Fellowship with Dr. Larry Hedges and Dr. Elizabeth Tipton. This fellowship will be a part of a new center which focuses on the development of statistical methods for evidence-based policy. This includes research on methods for […]

Estimating treatment effects on rates of rare events using precursor data: Going further with hierarchical models.

Someone points to my paper with Gary King from 1998, Estimating the probability of events that have never occurred: When is your vote decisive?, and writes: In my area of early childhood intervention, there are certain outcomes which are rare. Things like premature birth, confirmed cases of child-maltreatment, SIDS, etc. They are rare enough that […]