Sadish Dhakal writes: I am struggling with the problem of conditioning on post-treatment variables. I was hoping you could provide some guidance. Note that I have repeated cross sections, not panel data. Here is the problem simplified: There are two programs. A policy introduced some changes in one of the programs, which I call the […]

**Causal Inference**category.

## “Did Austerity Cause Brexit?”

Carsten Allefeld writes: Do you have an opinion on the soundness of this study by Thiemo Fetzer, Did Austerity Cause Brexit?. The author claims to show that support for Brexit in the referendum is correlated with the individual-level impact of austerity measures, and therefore possibly caused by them. Here’s the abstract of Fetzer’s paper: Did […]

## Causal inference with time-varying mediators

Adan Becerra writes to Tyler VanderWeele: I have a question about your paper “Mediation analysis for a survival outcome with time-varying exposures, mediators, and confounders” that I was hoping that you could help my colleague (Julia Ward) and me with. We are currently using Medicare claims data to evaluate the following general mediation among dialysis […]

## What if that regression-discontinuity paper had only reported local linear model results, and with no graph?

We had an interesting discussion the other day regarding a regression discontinuity disaster. In my post I shone a light on this fitted model: Most of the commenters seemed to understand the concern with these graphs, that the upward slopes in the curves directly contribute to the estimated negative value at the discontinuity leading to […]

## Another Regression Discontinuity Disaster and what can we learn from it

As the above image from Diana Senechal illustrates, a lot can happen near a discontinuity boundary. Here’s a more disturbing picture, which comes from a recent research article, “The Bright Side of Unionization: The Case of Stock Price Crash Risk,” by Jeong-Bon Kim, Eliza Xia Zhang, and Kai Zhong: which I learned about from the […]

## How to simulate an instrumental variables problem?

Edward Hearn writes: In an effort to buttress my own understanding of multi-level methods, especially pertaining to those involving instrumental variables, I have been working the examples and the exercises in Jennifer Hill’s and your book. I can find general answers at the Github repo for ARM examples, but for Chapter 10, Exercise 3 (simulating […]

## Causal inference: I recommend the classical approach in which an observational study is understood in reference to a hypothetical controlled experiment

Amy Cohen asked me what I thought of this article, “Control of Confounding and Reporting of Results in Causal Inference Studies: Guidance for Authors from Editors of Respiratory, Sleep, and Critical Care Journals,” by David Lederer et al. I replied that I liked some of their recommendations (downplaying p-values, graphing raw data, presenting results clearly) […]

## “Did Jon Stewart elect Donald Trump?”

I wrote this post a couple weeks ago and scheduled it for October, but then I learned from a reporter that the research article under discussion was retracted, so it seemed to make sense to post this right away while it was still newsworthy. My original post is below, followed by a post script regarding […]

## Did blind orchestra auditions really benefit women?

You’re blind! And you can’t see You need to wear some glasses Like D.M.C. Someone pointed me to this post, “Orchestrating false beliefs about gender discrimination,” by Jonatan Pallesen criticizing a famous paper from 2000, “Orchestrating Impartiality: The Impact of ‘Blind’ Auditions on Female Musicians,” by Claudia Goldin and Cecilia Rouse. We’ve all heard the […]

## Difference-in-difference estimators are a special case of lagged regression

Fan Li and Peng Ding write: Difference-in-differences is a widely-used evaluation strategy that draws causal inference from observational panel data. Its causal identification relies on the assumption of parallel trend, which is scale dependent and may be questionable in some applications. A common alternative method is a regression model that adjusts for the lagged dependent […]

## Do regression structures affect research capital? The case of pronoun drop. (also an opportunity to quote Bertrand Russell: This is one of those views which are so absurd that only very learned men could possibly adopt them.)

A linguist pointed me with incredulity to this article by Horst Feldmann, “Do Linguistic Structures Affect Human Capital? The Case of Pronoun Drop,” which begins: This paper empirically studies the human capital effects of grammatical rules that permit speakers to drop a personal pronoun when used as a subject of a sentence. By de‐emphasizing the […]

## 13 Reasons not to trust that claim that 13 Reasons Why increased youth suicide rates

A journalist writes: My eye was caught by this very popular story that broke yesterday — about a study that purported to find a 30 percent (!) increase in suicides, in kids 10-17, in the MONTH after a controversial show about suicide aired. And that increase apparently persisted for the rest of the year. It’s […]

## A debate about effect-size variation in psychology: Simmons and Simonsohn; McShane, Böckenholt, and Hansen; Judd and Kenny; and Stanley and Doucouliagos

A couple weeks ago, Uri Simonsohn and Joe Simmons sent me and others a note that they were writing a blog post citing some of our work and asking for us to point out anything that we find “inaccurate, unfair, snarky, misleading, or in want of a change for any reason.” I took a quick […]

## Continuing discussion of status threat and presidential elections, with discussion of challenge of causal inference from survey data

Last year we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz. A couple months later we updated with Mutz’s response to Morgan’s critique. Finally, Morgan has published a reply to Mutz’s response to Morgan’s comments on Mutz’s paper. Here’s a passage that is of methodological interest: […]

## “How many years do we lose to the air we breathe?” Or not.

From this Washington Post article: But . . . wait a second. The University of Chicago’s Energy Policy Institute . . . what exactly is that? Let’s do a google, then we get to the relevant page. I’m concerned because this is the group that did this report, which featured this memorable graph: See this […]

## Automatic voter registration impact on state voter registration

Sean McElwee points us to this study by Kevin Morris and Peter Dunphy, who write: Automatic voter registration or AVR . . . features two seemingly small but transformative changes to how people register to vote: 1. Citizens who interact with government agencies like the Department of Motor Vehicles are registered to vote, unless they […]

## Conditioning on post-treatment variables when you expect self-selection

Sadish Dhakal writes: I am struggling with the problem of conditioning on post-treatment variables. I was hoping you could provide some guidance. Note that I have repeated cross sections, NOT panel data. Here is the problem simplified: There are two programs. A policy introduced some changes in one of the programs, which I call the […]

## “Incentives to Learn”: How to interpret this estimate of a varying treatment effect?

Germán Jeremias Reyes writes: I am currently taking a course on Applied Econometrics and would like to ask you about how you would interpret a particular piece of evidence. Some background: In 2009, Michael Kremer et al. published an article called “Incentives to learn.” This is from the abstract (emphasis is mine): We study a […]

## Wanted: Statistical success stories

Bill Harris writes: Sometime when you get a free moment, it might be great to publish a post that links to good, current exemplars of analyses. There’s a current discussion about RCTs on a program evaluation mailing list I monitor. I posted links to your power=0.06 post and your Type S and Type M post, […]

## Active learning and decision making with varying treatment effects!

In a new paper, Iiris Sundin, Peter Schulam, Eero Siivola, Aki Vehtari, Suchi Saria, and Samuel Kaski write: Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action a to take for a target […]