Skip to content
 

Just when you thought it was safe to go back into the water . . . SHARK ATTACKS in the Journal of Politics

We’ve been here before.

Back in 2002, political scientists Chris Achen and Larry Bartels presented a paper “Blind Retrospection – Electoral Responses to Drought, Flu and Shark Attacks.” Here’s a 2012 version in which the authors trace “the electoral impact of a clearly random event—a dramatic series of shark attacks in New Jersey in 1916” and claim to “show that voters in the affected communities significantly punished the incumbent president, Woodrow Wilson, at the polls,” a finding that has been widely discussed in political science over the past several years and was featured in Achen and Bartels’s recent book, Democracy for Realists.

In 2016, Anthony Fowler and Andy Hall reanalyzed the data and concluded that “the evidence is, at best, inconclusive”:

First, we assemble data on every fatal shark attack in U.S. history and county-level returns from every presidential election between 1872 and 2012, and we find little systematic evidence that shark attacks hurt incumbent presidents or their party. Second, we show that Achen and Bartels’ finding of fatal shark attacks hurting Woodrow Wilson’s vote share in the beach counties of New Jersey in 1916 becomes substantively smaller and statistically weaker under alternative specifications. Third, we find that their town-level result for beach townships in Ocean County significantly shrinks when we correct errors associated with changes in town borders and does not hold for the other beach counties in New Jersey. Lastly, implementing placebo tests in state-elections where there were no shark attacks, we demonstrate that Achen and Bartels’ result was likely to arise even if shark attacks do not influence elections. Overall, there is little compelling evidence that shark attacks influence presidential elections, and any such effect—if one exists—appears to be substantively negligible.

Fowler and Hall published their article in the Journal of Politics, and Achen and Bartels replied with a comment of their own, including these points:

Attributing to us [Achen and Bartels] the notion that, in general, “shark attacks influence presidential elections,” much less that “irrelevant events generally influence presidential elections”, reflects a profound misreading of our argument. As we spelled out, the 1916 attacks were politically relevant because substantial economic losses ensued and the president was explicitly blamed. Neither of those things is true of the typical shark attack; thus, we would not expect it to matter at the polls. . . .

Fowler and Hall’s reanalysis of county-level voting patterns in New Jersey in 1916 produces “substantively smaller and statistically weaker” estimates of the impact of shark attacks on electoral support for Woodrow Wilson “under alternative specifications” . . . Even so, their point estimates mostly differ only modestly from ours. The estimates become statistically insignificant only because their ahistorical bad statistical fits inflate standard errors and thus make the t-statistics smaller. We show that a variety of regression models that get the politics right all fit better than Fowler and Hall’s. Those models all show a substantively and statistically significant shark effect. . . .

Fowler and Hall employ a series of “placebo tests” comparing election outcomes in coastal and noncoastal counties to suggest that “Achen and Bartels’s result for New Jersey in 1916 was somewhat likely to arise even if shark attacks have no effect on presidential elections” . . . They find that 27% of these comparisons produce “statistically significant” differences in the vote swing from one election to the next in counties bordering the ocean; hence, they argue that other factors besides shark attacks could have produced the marked electoral shift in Jersey Shore counties in 1916. But Fowler and Hall provide no indication of what those other factors might be. . . .

This last point seems to represent a statistical misunderstanding on the part of Achen and Bartels. Fowler and Hall’s point regarding the placebo tests—a point which I think is valid—is that correlations in the dataset make statistical significance easier to attain than would be expected under the simple default model of independent outcomes. Thus, the statistical significance of Achen and Bartels’s original analysis cannot be taken as strong evidence: the patterns they found could be explained by various systematic differences between counties unrelated to shark attacks.

Achen and Bartels are right that Fowler and Hall do not “demonstrate that the shark attacks made no difference.” That’s right. What Fowler and Hall find is there is no strong evidence for any effects of shark attacks.

Step back a minute. Before Achen and Bartels (2002, 2012), I assume that few scholars would’ve considered shark attacks to have important effects in 1916 or in any other presidential election. The Achen and Bartels papers made the surprising claim that, yes, shark attacks did make a difference, and they took this as evidence for their “blind retrospection” theory. As I wrote earlier when discussing all this, I think Achen and Bartels have some good points in their book; their larger arguments do not rely on the validity of that shark-attack study. Anyway, the point is that Fowler and Hall don’t need to demonstrate that shark attacks made no difference. The burden is on Achen and Bartels to support their counterintuitive statement that shark attacks mattered. Or, to put it another way, you can believe that shark attacks mattered in 1916, even if the data don’t really show it. All sorts of things could’ve mattered, and you can’t disprove any of them.

OK, fine. That all said, Achen and Bartels have two substantive points to make. The first is that irrelevant events can matter in presidential elections if “substantial economic losses ensued and the president was explicitly blamed.” The second is that Fowler and Hall’s estimates are similar to theirs, and that Fowler and Hall find non-statistical-significance only by fitting a crappy model and thus obtaining artificially high standard errors.

Now let’s turn to Fowler and Hall’s reply in that same journal.

I’ll pull out two parts of this reply.

First, regarding the effects of shark attacks:

[Achen and Bartels] agree with us that shark attacks do not, in general, lead voters to punish incumbents, stating that they “would not expect” the “typical shark attack” to affect a presidential election. This consensus is important since many readers have thought that their claims were stronger and more general than they are. . . . Writing in Pacific Standard, Seth Masket states that “voters punish their leaders for . . . shark attacks.” In his review of Achen and Bartels’s book in the Journal of Politics, Neil Malhotra writes that “voters frequently punish incumbents for things they cannot control such as shark attacks.” Achen and Bartels have now clarified that their claim is specific to only the shark attacks in New Jersey in 1916. . . . Future discussions of “blind retrospection” should take note of this new consensus. Rather than claiming that shark attacks indicate a general failure of electoral accountability, Achen and Bartels say that voters do not blame incumbents for shark attacks in about 99 percent of all recorded shark attacks in American history.

Well put.

Second, regarding the new analyses that give larger standard errors and thus diminish the claims of strong evidence regarding the 1916 election:

We show that the 1912 election, which Achen and Bartels use to control for the baseline political preferences of counties and towns, was anomalous. Figure 3 of our paper clearly shows that their county-level result is driven by the unusualness of 1912, not anything that happened in 1916. . . .

We show that Achen and Bartels’ standard errors are misleading. If we apply their inferential strategy to state-elections with no shark attacks, we detect an effect as large as theirs 32 percent of the time, and the estimate is statistically significant (p < .05) 27 percent of the time.

To see more on this, I recommend you go back and look at our earlier post, in particular this graph of adjusted data from Achen and Bartels:

screen-shot-2016-10-25-at-6-35-16-pm

and this graph of raw data from Fowler and Hall:

screen-shot-2016-10-29-at-9-07-53-am

I agree with Fowler and Hall that the evidence isn’t nearly as strong as implied by Achen and Bartels’s regressions.

Let me be clear: I don’t think that what Achen and Bartels did is any sort of scandal. They have an interesting idea regarding blind retrospection, the shark attack example is a cool case study, and yes their data are consistent with no effect of shark attacks but their data are also consistent with a positive effect, perhaps for the reasons they stated regarding economic costs and the president being blamed. They did some analyses which confirmed their beliefs and they published. Fair enough. Later, some other researchers looked at their data more carefully and found the evidence, both for this particular case and for shark attacks more generally, to not be so strong. That’s how we move forward.

Fowler adds:

Andy Hall and I wrote about this a bit in the conclusion of our sharks paper, but it’s very difficult to show evidence either way regarding the competence of rationality of voters. For one, lots of seemingly irrational behaviors have rational explanations. For example, it’s not necessarily irrational for voters to change their beliefs about their elected officials as a result of shark attacks–maybe the voters learned that the government doesn’t have their back when a major crisis comes their way. And even then, once you’ve adopted irrationality or incompetence as your “theory,” there’s nothing constraining your empirical testing. Maybe sharks affect people in the beach towns, maybe the beach counties, maybe the coastal counties, or maybe the whole state. You can run 10,000 different regressions and each one is just as (poorly) grounded in theory as the next, so all bets are off. And whichever regression gives you the desired result, you just say “well, that must be how irrational voting works.”

One fun solution would be for people like Chris Achen and Larry Bartels to get together with people like me and Andy Hall to come up with some ex-ante tests of rationality/competence/etc. We could all agree on some compelling tests that would partly adjudicate some of these debates and then go out and run the experiments or collect the relevant data.

P.S. I have no financial conflicts of interest here, but in the interests of full disclosure I should inform you that I’ve been involved in disputes with Larry Bartels before. In one dispute, Bartels and I were on the same side (this was in dealing with the annoying Thomas Frank); in the other case, we disagreed with each other. So, sometimes we agree, sometimes we don’t.

3 Comments

  1. Z says:

    Isn’t it generally accepted that (a) the president does not have that much impact on economic trends; and (b) the electorate votes against incumbents more when the economy is doing poorly? So if shark attacks really did disproportionately damage the economies of beach towns because of their reliance on tourism, isn’t it actually completely intuitive that there would be an (indirect) effect of shark attacks on the vote? It would be surprising if towns that were in economic hardship didn’t vote against the incumbent more than towns that weren’t in economic hardship. How large the effect should be and whether it would likely be detectable in the study by Achen and Bartels I have no idea.

    I had always heard the shark attack paper lumped in with college football losses (which don’t impact the economy significantly) as part of an argument that voters respond *directly* to negative emotions like fear or disappointment stemming from events having nothing to do with politics. Instead, it seems like the argument is that when people vote based on economic trends they (at least sometimes) don’t really consider the causes of those trends. I find this completely uncontroversial since economists often don’t know the causes of economic trends and people seem to definitely vote based on them. Fowler’s point that such voting might still be rational because it’s not based on punishing politicians for causing the trends but rather for not fixing them once they appeared is a good one. However, in many cases I would find this to be an example of irrational voting as well because (again) politicians don’t have that much control over the broader economy.

  2. Jonathan says:

    Imagine that we’re in Australia and there are shark attacks despite the precautions they take because they have more sharks and attacks: then you could say a voter might connect governmental failure to an event. When the mayor is in Las Vegas during a huge snowstorm, you can see how that could punish the mayor in the next election. There has to be a mechanism by which blame attaches. As a Jewish person, you know how that works: it can even be supernatural, the decoupling of the existence of actual Jews in places where there are no Jews so they become the shadowy, almost spirit-like causes of everything that you need to blame. In other words, the attachment mechanism can become irrational. And it clearly can become irrational within cultures, including Western cultures. One could say the Chinese Exclusion Act was an irrational attachment of fear, just as many today argue that restricting immigration out of fear of crime is irrational, but in all those you’re arguing about the extent to which something is irrational: mathematically, it resolves to rationality in some dimensions but not in others so you can come out on one side or the other depending on how you weight and slice the data.

    There must be an attachment mechanism or, at best, you’ve identified a correlation that lasts as long as it lasts. Like the history of your favorite team never losing in a bowl game though of course the players from early bowl games are long dead and the game is immensely different, so that’s a fun fact you hope stays true but it just represents a bunch of results that happened to come out this way, when affected, importantly, by the fact that they were in bowl games and thus had a chance of winning them (and you don’t count in your head all the years they didn’t play in a bowl game at all). To continue with the sports metaphor because I’m stuck with yet another Super Bowl hype experience here in Boston, that NE will now have been in half the Super Bowls over an 18 year period is not only remarkable but shows how impossible it is to detwine Belichick and Brady as causative factors. If you count the number of players that have passed through NE’s roster, if you count any measure, the odds that they could make 8 straight AFC Championship Games, that they could make 9 Super Bowls, that they could win 5 make analysis of why impossible: how do you take apart such an extreme thread of results? You could estimate that without Belichick, Brady might be like Drew Brees and make 1 or 2, but without Brady do you estimate Belichick might be at 4? Four Super Bowls without a start QB? Really? I’m assuming you don’t substitute Rodgers or Brees for Brady. I don’t see how you can figure that out; you can make arguments but you can’t reach a solid answer.

    What is the attachment mechanism for shark attacks? I can think of a few: government not caring about beach safety, government unable to provide enough employment so people are at the beach because they’re out of work. That second one is iffy as heck, but then Jews get blamed with much less rational attachment. I say real because you enter complex space through a rational value. In this light, the blaming of Jews is the taking of an irrationality and treating that as rational, slapping a label oven the gap to complete the thread. We see examples today on the ‘progressive’ side: if you define racism as institutional oppression, then you define Jews as white or non-white based on how you determine their support for institutional oppression, and that isn’t anti-Semitic to you because you aren’t reconciling that gap but are instead slapping a label over what you’re doing which hides the gap. The label inherits from the definition, which is why the anti-Semitic progressive faction pushes the definition so hard – though typically without stating that they are relying on a specific definition in which race hatred is defined institutionally. That fits the attachment mechanism because lots of people don’t want to believe they’re racist and nearly everyone wants to justify their own hatreds. This fascinates me not because I’m Jewish but because we can see and predict the pathways so well: if you push at them about their hatreds, they raise the stakes. So criticism of Farrakhan becomes Jews deserve criticism because they are white oppressors and they are white oppressors no matter what they do because they support Israel and that not only ‘trumps’ whatever good they do but becomes ‘they only do good because that covers up the evil they do as oppressors’. This fits every attachment narrative of grievance through history.

    I don’t mean to spend time on anti-Semitism for no reason. My point is that you can identify attachments because things do need to connect. You can identify where these attachments are rational and where they apply labels up or down: so the concept of ‘intersectionality’ – which has changed massively in the last few years – is harnessed to say that all ‘non-white’ peoples are oppressed, which includes any person in a ‘suspect’ category like T or Q, and that division into types flows across the data field so Jews end up providing the medicine necessary for a trans person to alter gender but that’s portrayed in the inverse, as an attempt to hide the fact that they’re oppressors. Or as I noted today an op-ed in the Forward that inverts the concept of how nation states don’t need to claim they exist into ‘Israel has no right to exist because it claims it has a right to exist and real nation states, like Switzerland don’t need to make this claim’, which completely ignores the fact that Israel responds to people questioning its right to exist. This kind of inversion of meaning becomes possible when you have attachments that enable threads to impose inherited meanings in different orderings. You see this all the time in math, with threads that flip meaning. In any programming, you can develop a value that becomes a reading of T or F. In set theoretical terms, they set the type and organize the data set according to type even when that clashes with actualities, so you end up with Jews do so much only to hide what they really do, which is only a step away from blaming the concept of ‘Jew’ for whatever is wrong in the world.

    The attachment doesn’t need to be much, which is the attraction I suppose of shark attacks affecting elections. But I think you make the point well: the most you should see in that case is a lot of ‘well, maybe if you tilt your head the right way’. Shark attacks aren’t a great choice because the numbers are really low. A better attachment would be if there is a serious case that acts as the attachment, like a shark net with a hole in it that lets a child be eaten off the beach. Then any shark sighting becomes a reminder of the failure. But that’s another form of modeling, one which defines not merely regressions but which assigns a value to an event and then looks at other events as supporting material. But that fits it to more standard polling stuff, doesn’t it? Like will the Pocahontas crap matter? How much did the Willie Horton ad actually matter? Those examples have attachments to threads that voters can follow in their minds: do I really identify and trust Liz Warren? To me, for example, the Willie Horton ad wasn’t about race but about Mike Dukakis as technocrat, so the ad used a vicious criminal and called on race but it formed an attachment to Mike as the kind of guy who processes things. By the time Obama ran, the concept had become that he was cool, that he kept his head, but Mike really is more a guy who processes stuff and so he couldn’t adequately counter that he keeps his head. He was unable to speak to the issue on an emotional level the way Obama could. (Aside: I knew him a little personally and don’t mean anything negative.)

    The tl/dr is: without attachment, expect maybe correlation at best, and if you see any appearingly meaningful level of correlation that’s a clue to look at specific events to find how they attached to the minds of voters.

  3. yyw says:

    There’s simply no way you can demonstrate causal effect of shark attacks based on historical data. If they made a prediction based on historical data and the prediction was borne out multiple times, then I would understand the obsession over such a study.

Leave a Reply