Skip to content
Archive of posts filed under the Causal Inference category.

A debate about effect-size variation in psychology: Simmons and Simonsohn; McShane, Böckenholt, and Hansen; Judd and Kenny; and Stanley and Doucouliagos

A couple weeks ago, Uri Simonsohn and Joe Simmons sent me and others a note that they were writing a blog post citing some of our work and asking for us to point out anything that we find “inaccurate, unfair, snarky, misleading, or in want of a change for any reason.” I took a quick […]

Continuing discussion of status threat and presidential elections, with discussion of challenge of causal inference from survey data

Last year we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz. A couple months later we updated with Mutz’s response to Morgan’s critique. Finally, Morgan has published a reply to Mutz’s response to Morgan’s comments on Mutz’s paper. Here’s a passage that is of methodological interest: […]

“How many years do we lose to the air we breathe?” Or not.

From this Washington Post article: But . . . wait a second. The University of Chicago’s Energy Policy Institute . . . what exactly is that? Let’s do a google, then we get to the relevant page. I’m concerned because this is the group that did this report, which featured this memorable graph: See this […]

Automatic voter registration impact on state voter registration

Sean McElwee points us to this study by Kevin Morris and Peter Dunphy, who write: Automatic voter registration or AVR . . . features two seemingly small but transformative changes to how people register to vote: 1. Citizens who interact with government agencies like the Department of Motor Vehicles are registered to vote, unless they […]

Conditioning on post-treatment variables when you expect self-selection

Sadish Dhakal writes: I am struggling with the problem of conditioning on post-treatment variables. I was hoping you could provide some guidance. Note that I have repeated cross sections, NOT panel data. Here is the problem simplified: There are two programs. A policy introduced some changes in one of the programs, which I call the […]

“Incentives to Learn”: How to interpret this estimate of a varying treatment effect?

Germán Jeremias Reyes writes: I am currently taking a course on Applied Econometrics and would like to ask you about how you would interpret a particular piece of evidence. Some background: In 2009, Michael Kremer et al. published an article called “Incentives to learn.” This is from the abstract (emphasis is mine): We study a […]

Wanted: Statistical success stories

Bill Harris writes: Sometime when you get a free moment, it might be great to publish a post that links to good, current exemplars of analyses. There’s a current discussion about RCTs on a program evaluation mailing list I monitor. I posted links to your power=0.06 post and your Type S and Type M post, […]

Active learning and decision making with varying treatment effects!

In a new paper, Iiris Sundin, Peter Schulam, Eero Siivola, Aki Vehtari, Suchi Saria, and Samuel Kaski write: Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action a to take for a target […]

What sort of identification do you get from panel data if effects are long-term? Air pollution and cognition example.

Don MacLeod writes: Perhaps you know this study which is being taken at face value in all the secondary reports: “Air pollution causes ‘huge’ reduction in intelligence, study reveals.” It’s surely alarming, but the reported effect of air pollution seems implausibly large, so it’s hard to be convinced of it by a correlational study alone, […]

“Heckman curve” update: The data don’t seem to support the claim that human capital investments are most effective when targeted at younger ages.

David Rea and Tony Burton write: The Heckman Curve describes the rate of return to public investments in human capital for the disadvantaged as rapidly diminishing with age. Investments early in the life course are characterised as providing significantly higher rates of return compared to investments targeted at young people and adults. This paper uses […]

Treatment interactions can be hard to estimate from data.

Brendan Nyhan writes: Per #3 here, just want to make sure you saw the Coppock Leeper Mullinix paper indicating treatment effect heterogeneity is rare. My reply: I guess it depends on what is being studied. In the world of evolutionary psychology etc., interactions are typically claimed to be larger than main effects (for example, that […]

“The Long-Run Effects of America’s First Paid Maternity Leave Policy”: I need that trail of breadcrumbs.

Tyler Cowen links to a research article by Brenden Timpe, “The Long-Run Effects of America’s First Paid Maternity Leave Policy,” that begins as follows: This paper provides the first evidence of the effect of a U.S. paid maternity leave policy on the long-run outcomes of children. I exploit variation in access to paid leave that […]

How to approach a social science research problem when you have data and a couple different ways you could proceed?

tl;dr: Someone asks me a question, I can’t really tell what he’s talking about, so I offer some generic advice. Joe Hoover writes: An issue has come up in my subsequent analyses, which uses my MrsP estimates to explore the relationship between county-level moral values and the county-level distribution of hate groups, as defined by […]

Postdoc in Chicago on statistical methods for evidence-based policy

Beth Tipton writes: The Institute for Policy Research and the Department of Statistics is seeking applicants for a Postdoctoral Fellowship with Dr. Larry Hedges and Dr. Elizabeth Tipton. This fellowship will be a part of a new center which focuses on the development of statistical methods for evidence-based policy. This includes research on methods for […]

Estimating treatment effects on rates of rare events using precursor data: Going further with hierarchical models.

Someone points to my paper with Gary King from 1998, Estimating the probability of events that have never occurred: When is your vote decisive?, and writes: In my area of early childhood intervention, there are certain outcomes which are rare. Things like premature birth, confirmed cases of child-maltreatment, SIDS, etc. They are rare enough that […]

Fitting multilevel models when the number of groups is small

Matthew Poes writes: I have a question that I think you have answered for me before. There is an argument to be made that HLM should not be performed if a sample is too small (too small level 2 and too small level 1 units). Lot’s of papers written with guidelines on what those should […]

New estimates of the effects of public preschool

Tom Daula writes: You blogged about Heckman and the two 1970s preschool studies a year ago here and here. Apparently there are two papers on a long-term study of Tennessee’s preschool program. In case you had an independent interest in the topic, a summary of the most recent paper is here, and the paywalled paper […]

Principal Stratification on a Latent Variable (fitting a multilevel model using Stan)

Adam Sales points to this article with John Pane on principal stratification on a latent variable, and writes: Besides the fact that the paper uses Stan, and it’s about principal stratification, which you just blogged about, I thought you might like it because of its central methodological contribution. We had been trying to use computer […]

No, I don’t buy that claim that Fox news is shifting the vote by 6 percentage points

Tyler Cowen writes: This is only one estimate, from Gregory J. Martin and Ali Yurukoglu, but nonetheless it is backed by a plausible identification stragegy and this is very interesting research: We find that in a hypothetical world without Fox News but with no other changes, the Republican vote share in the 2000 election would […]

The butterfly effect: It’s not what you think it is.

John Cook writes: The butterfly effect is the semi-serious claim that a butterfly flapping its wings can cause a tornado half way around the world. It’s a poetic way of saying that some systems show sensitive dependence on initial conditions, that the slightest change now can make an enormous difference later . . . Once […]