The effects of U.S. military aid on political violence in Colombia: a back-and-forth regarding the strength of the causal evidence

Following my comments on their article on U.S. military funding and conflict in Colombia, Oeindrila Dube and Suresh Naidu wrote:

Thanks for the comments on our paper. It seemed that you viewed the correlations in the anaysis as an interesting descriptive exercise, but not interpretable as causal. We agree with you that the most interesting social science is often causal, and in this case in particular the causal claims are the main results. The paper’s punchline is that military aid needs to be reconsidered when there is collusion between the army and non-state armed groups, and we couldn’t make this claim if we thought the results were purely descriptive.

In the paper, we do a lot of sample splitting and parametric time controls to rule out the possibility that this is a spurious effect. For example, our results are robust to including a base-specific time trend, along with a base-specific post-2001 dummy.

Possibly the best evidence against a strict “conflict” time-series interpretation is that there is no effect (positive or negative) of US military aid on guerrilla attacks near Colombian military bases. In other words, its not just an increase in conflict on all sides, but an increase in paramilitary attacks in particular.

The “differential time trend” that could drive our effect would have to be a) steeply nonlinear b) only applicable to paramilitaries in base municipalities, and c) would have to be fairly unique to the base municipalities, given the wide variety of alternate control groups we examine. So we think this is not a likely alternative explanation that can account for the effects.

To which I replied:

First off, I still would prefer associational language followed by causal speculation. But I can respect your different choice of emphasis. Now to get to details: my basic alternative model goes as follows:
– Conflict in Colombia increased during the early 2000’s.
– U.S. military aid, in the U.S. and elsewhere, increased during that period also.
– Most of the paramilitary attacks (and, thus, most of the increase in paramilitary attacks) occurred near military bases.
Thus, I’m not so impressed by the “differential time trend” argument. It’s unsurprising (but nonetheless worth noting, as you do) that there are fewer guerilla attacks near military bases. But that doesn’t mean that the paramilitary attacks wouldn’t have increased in the absence of U.S. aid.

None of the above really contradicts your main political story, which is that the Colombian military is involved in paramilitary attacks, and that U.S. aid is an enabler for this sort of violence.

My story above is consistent with your causal story–more U.S. aid, more resources for the military, more paramilitary attacks. It’s also consistent with a different causal story, which goes like this: more conflict, more paramilitary attacks, also more U.S. aid which actually serves to stop the situation from getting worse. The argument is, yes, the U.S. is giving weapons to the bad guys, but by doing so, it co-opts them and restrains their behavior.

OK, I’m not saying this latter argument is true, but I think your strongest argument against it is to say something like: “Sure, it’s possible that things would be getting even worse in the absence of U.S. military aid. But given that, during the time that aid was higher, violence was also higher–and we’re talking here about violence being done by the allies of the recipients of the aid–well, maybe aid isn’t such a good idea.” That is, you can put the burden of proof on the advocates of aid. Hey, it costs money and it’s going to some unsavory characters. You shouldn’t have to prove that aid is hurting; I think it would be more defensible, from a statistical/econometric point of view, to show the association and put the ball in their court.

P.S. Just to be clear: I don’t have any strong feeling that you’re wrong or any goal of “debunking” your paper. It’s interesting and important work and I’m trying to understand it better.

And then they shot back with:

Regarding the stylistic point about associations and causal claims, we think this is perhaps discipline-specific, as the style in economics seems to be to make a causal claim and then rule out all the alternative causal stories as much as possible. I’m sure this is probably one of many idiosyncrasies that irks non-economists.

The substantive question is why paramilitary attacks (and paramilitary attacks specifically, rather than other measures of conflict), increase more in places near bases. The account we put forward is that this occurs because the Colombian military funnels a share of its resources to paramilitary groups. Thus, if US military aid translates into more resources for the military which are shared with paramilitary groups, the implication is that in the absence of increases in US military aid, paramilitary attacks would not have increased by as much as they did.

Now the alternative account you put forward is “more conflict, more paramilitary attacks, also more U.S. aid which actually serves to stop the situation from getting worse. The argument is, yes, the U.S. is giving weapons to the bad guys, but by doing so, it co-opts them and restrains their behavior.”

It seems like you have two distinct things in mind, that overall conflict is a source of bias, and an associated conjecture that this omitted variable (overall conflict) upward biases our main coefficient since it is positively correlated with paramilitary attacks and positively correlated with the aid shock. First, we explicitly address and rule out potential omitted variables using a number of empirical specifications. But, even if there is an omitted variable correlated with U.S. military aid that differentially affects paramilitary attacks in base municipalities, it is not clear whether the direction of the bias would be positive. As an example, say a change in Colombian government leads the state to become more effective in fighting the guerilla insurgency, and the US rewards the state with more military aid, while paramilitary activity declines differentially in base regions, as this activity becomes less necessary with greater military effectiveness. In this case, the omitted variable (stronger Colombian state) is negatively correlated with paramilitary attacks and positively correlated with the aid shock, and this would lead us to underestimate the true effect of U.S. aid on paramilitary activity.

Moreover, we think we do a good job ruling “conflict in general” at the national, state, or municipality level as a confounding variable. “Overall conflict” variation at the country level is absorbed by year fixed effects, and conflict at the department level is absorbed by the department x year fixed effects. At the municipal level, it is NOT the case that we observe increases in overall conflict, such as total number of clashes amongst all armed actors at the municipal level. (In out data, attacks are one-sided events carried out by a particular group. The fact that we see paramilitary attacks increase means we are specifically observing increases in events that involve only paramilitary groups – e,g, the paramilitaries attack a village or destroy some type of infrastructure. ) Also, in every specification we find no effect on the guerrilla attacks, and we think you are not taking the non-effect sufficiently seriously in terms of countering the overall conflict account. The guerilla non-effect actually provides very robust evidence that the U.S. military aid is not just correlated with any type of conflict, but rather with attacks by a particular group (which has no regional spillovers).

In addition, our base-specific linear trend and post-2001 dummy specification should convince you that our effect is not merely a post-2001 increase in conflict that manifests particularly as paramilitary attacks in base municipalities.

Your alternative account suggests that more aid to paramilitary organizations could actually result in less violence. While it is challenging to know what the counterfactual would have been in the absence of increased aid, Figure 2 shows that when aid rises sharply in 1999 there is a differential increase in aid in the base regions, and when aid decreases in 2001, there is a corresponding closing of differential decrease in the base regions. This seems inconsistent with the idea that lower aid translates into more paramilitary activity. Also, after 2002, when aid rises again, the differential increases yet another time. It is difficult to explain this pattern with the account you put forward, which would have to require additional coincidental reasons why paramilitary attacks should increase more in base regions precisely in 1999, then decline in 2001, and then rise again in 2002. This is possible, but seems unlikely.

We were thinking of some ideas that would be consistent with your alternative account, of why more aid to paramilitary organizations could actually lower violence. One story here could be deterrence – that stronger paramilitaries deter the guerillas resulting in fewer attacks by guerillas or fewer clashes between guerillas and paramilitaries. But, our results do not show a fall in guerilla attacks or clashes amongst the two groups; rather the coefficient on these other variables is close to 0 and they are statistically insignificant, which is inconsistent with the deterrence account.

Another reason could be dependence, that in the short run U.S. aid increases paramilitary violence, but it also induces paramilitary reliance on the Colombian military for supplies, which increases the sway the government has vis-à-vis this group, potentially leading to future demobilization. Thus in the long-run, U.S. military aid reduces paramilitary violence. While this process could take “long and variable lags” to manifest, it is important to note that we see a dramatic increase in paramilitary activity in 2005, despite a half-decade of huge U.S. military transfers to Colombia. Thus we do not see evidence of this dependence account in our data.

2 thoughts on “The effects of U.S. military aid on political violence in Colombia: a back-and-forth regarding the strength of the causal evidence

  1. Absent all the wrangling about whether the causal arrows in this study are drawn correctly, one massive underlying question would still remain: are the data *correct*, in the sense of accurately representing micro patterns of violence in Colombia? I'm willing to bet that data on US aid is…decent. Maybe. I'm absolutely unwilling to bet that the data on violence are of similar quality. It's both technically simpler and more in line with the publish-or-perish incentive to be naive about these data, but pretending they're as simply countable as dollars, votes or survey responses just isn't reasonable.

    Violence data are certainly "incorrect" in the sense of invariably failing to report each and every episode or casualty of violence. Violence data are highly likely also to be incorrect in the sense of having extremely uneven reporting rates across extremely relevant strata (time, location, victim groups). In the case of this paper, neither the underlying dataset nor the authors (as far as I can tell) make any attempt to deal with the problems of descriptive inference, as opposed to causal inference, inherent here.

    The paper's authors describe CERAC's methods of "cross-checking" data with other sources, but significant (and unevenly distributed!) chunks of Colombia's violence remain unreported. There are groups in rural Colombia who will not report to a representative of the church. Many people don't want to report to the police. My colleagues and I at the Human Rights Data Analysis Group have done some looking at data coverage problems in Colombia (here: http://hrdag.org/about/colombia.shtml).

    Of course, the study's descriptive conclusions could still be correct. Or it could be that areas around bases are targeted by human rights watchdogs for increased reporting. Or it could be that areas around bases could be off-limits to human rights watchdogs. Or it could be that human rights watchdogs, governmental and non, have budgets for reporting that vary from year to year. There are very few good ways to know; I certainly don't. In any case, my usual plea in the case of papers that exploit micro variation in violence is: see what sort of confidence interval you could attach to your descriptives, before you engage in causal — or even associational — debate.

  2. Some of us economists, too, are irked by our discipline's tradition of hard-selling a causal claim, then (pretending to) rule out (a selected set of) alternative explanations.

Comments are closed.