Darren Grant writes:

Thanks for bringing my work on ballot order effects to the attention of a wider audience via your recent blog post. The final paper, slightly modified from the version you posted, was published last year in Public Choice.

Like you, I am not wedded to traditional hypothesis testing, but think it is the right way to go here. The post spoke to the issue that I struggled with the most in writing the paper—the role of description. I view this as an important aspect of an empirical analysis, and normally should be done early on. This paper was unique, for me at least, in that I chose to invert this order—twice.

The first time involved the presentation of mean vote shares by ballot position—the best simple way to describe this data, in my opinion. Instead of beginning with these, I began with regression results, and then presented the (differences in) means as a robustness check. As discussed in the paper, these means check for bias caused by imperfect randomization, as well as (hopefully) reassure the reader that the more complex SUR method isn’t driving the results.

The second time involved the magnitude of effect, which as Dale points out is quite large. I support the plausibility of effects this large in the “money shot” figure, Figure 2a, that comes near the end of the paper. This simple histogram simply depicts one candidate’s county-wide vote shares in an unusual two-candidate contest (described on p. 21 of your version of the paper):

The histogram has two peaks, at 40% and 60% of the vote. These correspond to second/first ballot position, and imply a ballot order effect of roughly 20 percentage points—at least double the main estimates presented earlier in the paper (which top out around 10 percentage points). Regression estimates (Figure 2c) also indicate a (nearly) 20 percentage point ballot order effect.

I am not adamant that this order of presentation was correct—just wanted to share with you the way I tried to use description in a non-standard way to alleviate some of the concerns you raised about the estimates. Upon request, I can address some of the other points that were raised.

I have three comments:

1. I have not looked at this paper in detail, but I guess if there are large ballot order effects, I’d expect to see them in these sorts of low-information races. (Recall my skepticism about the claim that Trump won the presidential election in 2016 because of ballot order in key states.)

2. I don’t see the point of hypothesis testing here at all. Nobody thinks the ballot order effect is exactly zero.

3. Figure 2a confuses me. If you want to show vote percentages with the two different ballot orders, why not show two histograms, one where this candidate is first on the ballot and one where he’s second? That would seem to be the more direct comparisons. Also, what’s with the bizarre x and y axes here? Better would be to put x-axes at 30, 40, 50, 60, 70, and y-axes at 0, 5, 10.