Exogenous tax cuts and treatment interactions

Nate Silver and Greg Mankiw have an interesting exchange about the use of exogenous instruments to estimate causal effects. Unfortunately, the subject is macroeconomics, a topic on which I know next to nothing beyond what I learned in Mr. Cutlip’s econ class in 11th grade. But I think it is, in Greg’s phrase, “a teachable moment” on the subject of causal inference.

Greg summarizes the exchange pretty well, although I think he’s missing a key point.

Nate noticed a newspaper article where Greg related research by Christina and David Romer on the effects of “exogenous” tax cuts on the economy. Nate writes:

The type of tax cut that Romer and Romer think falls into this category is what they call an “exogenous” tax cut — one designed not to counter business cycles, but rather a “spontaneous” tax cut under relatively healthy economic circumstances.

This is very much not the type of tax cut that we are contemplating right now. Instead, what is being contemplated is a countercyclical action in an unhealthy economy designed to return the economy to normal growth. Romer and Romer are not all that keen on this type of tax cut; in fact, they argue that such “countercyclical fiscal policy is not achieving its intended purpose” . . .

Greg repiies:

Why did the Romers focus on exogenous policy changes? The reason is that these are the only changes that can be used to reliability identify the effects of tax policy. . . . The Romers focus on exogenous tax changes for the same reason doctors conduct randomized drugs trials–not because they are interested in randomization as a prescriptive tool, but because randomization solves a statistical identification problem.

And now here are my thoughts, again with full recognition that I can really only comment on the statistical issues here, not the economics.

First, Greg is right that it is generally considered desirable or even optimal to estimate treatment effects using randomized experiments or exogenous implementations (but see here for an opposite view from James Heckman), even when the ultimate goal is to understand how the intervention works in the wild, so to speak.

But there is the potential for treatment interactions–that is, a treatment might be more effective in some conditions than in others. There’s lots of evidence for treatment interactions in various settings, ranging from education to job training. And this is what Nate is talking about. Again, without attempting to comment on the economics, the treatment effect could vary enough that Nate could be right about the direct relevance of the Romers’ study of exogenous tax changes.

To put it another way, Greg is talking about identifiability and Nate is talking about generalizability.

Greg writes, “I usually don’t respond to blogosphere commentary on my work because, after all, time is scarce.” But since he’s had time to respond once, perhaps he’ll be able to respond again and clarify this issue. (I think my time is particularly non-scarce since I’m responding to blogosphere commentary on somebody else’s work!) In any case, I like the idea of shifting the debate to a discussion of treatment interactions since then it might be more possible to resolve this on a technical level. Perhaps a teachable moment for me as well as for others.

5 thoughts on “Exogenous tax cuts and treatment interactions

  1. Except that the Romers don't seem to believe there is any parameter variation. There's nothing very complex in either the economics or the econometrics in the paper. The main point of the paper is stated clearly on the bottom of p. 21: "First, changes in the level of taxes have large
    effects on economic activity: following tax changes undertaken for reasons largely unrelated to other influences on output, there are large and significant output movements in the opposite direction. Second,
    how one measures tax changes matters: using broader measures substantially obscures the impact of tax changes on the economy." "Broader measures" means combining exogenous and endogenous tax policy changes; the latter suffer from simultaneity bias. Silver just didn't understand the paper.

    I must really have a lot of time to waste if I'm posting comments on comments on comments…

  2. You may be right that Nate Silver confused identifiability and generalizabilty. The word "exogenous" comes up in discussions of identification, so the post at my blog spoke to that issue.

    What about the generalizability? In my original NY Times articles on which Mr Silver was commenting, I wrote:

    "Why this is so remains a puzzle. One can easily conjecture about what the textbook theory leaves out, but it will take more research to sort things out. And whether these results based on historical data apply to our current extraordinary circumstances is open to debate."

    So I think I anticipated the issue of generalizability and flagged it for the reader. Very little empirical research, as far as I know, sheds much light on the matter.

  3. I suspect the claim that Silver "didn't understand the paper" is claiming more than can be known, given what Silver has written. Even if we allow that Mankiw and Silver begin in different places, identification and generalizability, where Silver ends up is with Mankiw's claim in a press article. In that article, Mankiw did exactly what Silver says he did. Mankiw offered the 3 multiplier when discussing a tax cut intended to offset weak economic activity. The Romers attached the 3 multiplier to cuts that are, by definition, not intended to offset weak economic activity. That is what Silver criticized. How does that represent a misunderstanding of the Romer paper on Silver's part?

  4. Greg,

    I don't know anything about this particular example but I certainly agree that generalizability is tricky, if for no other reason than statistical power issues–interactions are generally estimated with less precision than main effects, which is particularly a problem given that interactions are typically smaller than main effects.

    It's just a funny thing how in statistics and econometrics, there's so much discussion of different sorts of "average treatment effects" and not always much discussion about the variation that is being averaged over. Rajeev Dehejia's work in this area is helpful, I think.

  5. You might be interested in a recent paper in Political Analysis, that deals exactly with the issue here. Some might have missed it because the title is kinda generic ("Model Specification in Instrumental-Variables Regression".) Namely, the paper deals with this question: is the causal effect of exogenous variation in the treatment induced by the instrument (i.e., the effect that IV models estimate) the same as the causal effect of the treatment?

    http://pan.oxfordjournals.org/cgi/content/abstrac

Comments are closed.