Two of my political science colleagues (Bob Shapiro, an expert on polling and public opinion, and Bob Erikson, an expert on campaigns and elections) wrote something for the Columbia University newsletter about the recent 2021 elections:
Two notable things happened in the recent 2021 elections. First, there was a Republican swing in New Jersey, Virginia, and elsewhere. Second, the polls were off in New Jersey, with the Democrat receiving only about 51% of the two-party vote, after being at about 54% in the polls. A three percentage point polling error is hardly unprecedented, but it’s on the high end of what pollsters like to see.
As students of political science and statistics, we would emphasize two aspects of the 2021 Virginia and New Jersey governor elections. They both have to do with the surprise—given what was expected in pre-election polling—at how well Republicans did, especially in New Jersey, and raise the question: How much can we count on the polls to provide an accurate baseline?
In 2016 and 2020, the presidential election polls revealed problems in underestimating the Republican vote in key states, as described in an important report by the American Association for Public Opinion Research. But for Virginia and New Jersey, the 2020 pre-election Real Clear Politics polling averages for president were within two percentage points of the actual results. Further, the average for the 2017 governor election polls in New Jersey was within one percentage point for Democrat Phil Murphy’s victory. In the 2017 Virginia election, the polling average underestimated Democrat Ralph Northam’s victory by five points—an underestimate of the Democrat, not the Republican.
How the New Jersey Polls Differed From Virginia
How did the Virginia and New Jersey polls do in 2021? There were not many of them, but based on the Real Clear Politics polling average, the Virginia polling was on target: the poll average and the final results both had Republican Glenn Youngkin winning by two points.
New Jersey was different. There were similarly few polls in the week before the election, and they were substantially off the mark.
Polling errors are of interest to journalists, political junkies, and polling professionals. A national partisan swing—also reflected in the president’s dropping approval rating—is important to just about everybody, given the sharp differences between the two parties in issues ranging from unemployment benefits and corporate taxes to energy and environmental policy to immigration, abortion, and policing.
While the New Jersey polling was a surprise, the overall election result was less so to us. And we say this without discussing what the pundits have lamented over for the Democrats: that the Biden administration’s problems hurt Democratic candidates and the hair-pulling over campaign issues had an impact on elections in Virginia and New Jersey. Based on historical trends, the party that has just won the White House almost never wins the New Jersey or Virginia governorship the following year. All presidents—Democrats and Republicans, popular and less so—have faced this deflation of their party’s support. The question is why.
Centrist Voters and Ideological Balance
Some claim that centrist voters tend to seek an ideological balance of their government officials. American politics is polarized, but some swing voters remain. The Democrats currently control two of the three branches of the national government (Congress and the presidency), and it makes sense that swing voters will want to balance that by voting Republican in the election that immediately follows. Some identify the reason to be that voters for the losing side in the presidential race are more motivated to vote in subsequent elections. These same forces are likely to continue in the 2022 midterm elections. Historically, the president’s party almost always loses seats in the midterm election.
The swinging pendulum explains the vote but not Biden’s declining approval ratings, which we can attribute to bad news from Afghanistan, the continuing disruption caused by Covid, and a sluggish economy. But most presidents have seen a decline in approval during their first year in office.
A second set of explanations for the election loss are more specific to the actions of Biden and others in the Democratic party. From the left, the argument is that voters are slamming the Democrats for being ineffectual and not delivering on their campaign promise of economic relief. From the right, the argument is that, now with Trump out of the way, moderate voters can see leftism in all its glory—and they don’t like it.
One way to sharpen this distinction is to imagine what would have happened had Biden led his party to enacting a large spending bill a few months ago as originally planned. On the one hand, this would have represented an argument in favor of the Democrats as a party that gets things done. On the other, this could have served as motivation for the pendulum to swing back—even if each of the items on the bill were broadly popular. Indeed, had a big bill passed, this could be blamed for the recent mediocre economic news. Looking back, perhaps had Biden passed a truly centrist bill with Republican support, this could have helped the Democrats in the election. The president is looking toward 2024, and a stimulus that begins now could lead to the economy springing back in the next three years. Political science research suggests that economic trends during the election year can determine the winner.
Were We Too Influenced by the Georgia Runoff in January?
The two explanations—pendulum and policy—can be fit together, but they differ in their implications. Given our political science background, we are inclined to go with the pendulum theory, but in that case why was the political science profession not telling everyone to expect a Republican swing in 2021? It may be that we were too influenced by the Georgia runoff elections in January, which were won by two Democrats: a counterexample to the usual expectation of a swing away from the party in power and perhaps explainable by continuing anger at Trump’s refusal to accept the November election outcome.
If we look back at 2020, we can see that the GOP held its own apart from Trump’s defeat. The 2020 election results foreshadowed the 2021 Republican gains. While Trump was losing the presidency in 2020, the Republicans still picked up 13 seats in the House of Representatives and maintained their control of 30 state legislatures compared to the Democrats’ 18.
Without Trump on the ticket, the Republicans may find themselves better off. The Democrats will be confronted by this challenge in the 2022 midterm elections, which will be a referendum on the performance of the Biden administration and the Democrats in control of Congress, especially regarding the Covid-19 pandemic and the national economy.
I guess the question is, if a close Democratic win in New Jersey and a close Republican win in Virginia was really no surprise, and should’ve been no surprise to anticipate six months ago, why was it all seeming like such a surprise a few days ago? And this brings us back to the polls. For months, the Democratic candidates had a narrow but consistent polling lead in Virginia and a moderate and consistent lead in New Jersey. In this era of political polarization and stable polls, it’s natural to think that a consistent lead will persist. So maybe the point is that, in the months before the election, journalists and political scientists tended to take the polls at face value—after all, they did represent a small swing of the pendulum toward the Republicans. But in retrospect, they—we—should’ve been saying that the polls indicated a smaller swing than would’ve been expected based on past elections. And that reasoning would’ve led to some partial pooling and more of a recognition of the range of plausible election outcomes.
I don’t think journalists and political scientists will make this mistake in 2022. Why? Not just cos they were burned so will be more careful. No, I’m saying this because we’re used to predicting the midterm elections given incumbency and trial-heat polls. For example, back in February, 2010, I wrote, “The Democrats are gonna get hammered.” This was nearly nine months before the election, but we knew what was coming. We don’t have so much data on the odd-year races, also these elections are less consequential, so it was easy for everyone to be a bit more sloppy in their mental accounting of polls and predictions.
P.S. More on NJ here from Eric Lach.
I think sentences like this are confusing: “Looking back, perhaps had Biden passed a truly centrist bill with Republican support, this could have helped the Democrats at the polls.” The post is about political polls, not the election polls. All of the reasons why R or D might do better or worse is really off the point – the point was why the polls were off. Suppose there was an extreme rejection of one party or the other – that, in itself, means nothing for the accuracy of the polls. They could have picked up a dramatic change in public sentiment. So, I find the mixing of reasons why the Democrats did not do as well as expected and Republicans did better than expected confusing – I don’t see why those reasons say anything about the accuracy of the polls.
Dale:
By “at the polls,” I mean the election. I changed to clarify.
I’m still confused. The post is laden with reference to polls and errors in polling. How or why is the discussion about voter sentiment regarding Republicans and Democrats related to the polling errors?
What I found interesting about the post is that it wasn’t technical. Pollsters, from what I understand, depend upon a randomized process of selecting potential voters, from a database, and contacting them by phone (cell or land).
There are, I know, a few firms that use social media for “whisper polls”. That is, secretly monitoring potential voters, sampling, and then forming a distribution of who is likely to vote for which candidate. But, I think this is in the (deep) minority.
In my very uninformed opinion, polling error is correlated with its methodology. That is, until a statistical methodology is developed for social media, these polling errors are reflective of a) Americans don’t trust receiving anonymous calls b) Americans are more comfortable expressing unbiased opinions to strangers on social media.
Sam:
Some pollsters use internet panels.
Clearly, it’s the “shy Trump voter that” explains why Republicans outperformed the polling in the 2021 elections….
Just like in 2016 and 2020, the “shy Trump voter” explained why Republicans outperformed the polling.
Oh, wait….What’s that you say?
Trump wasn’t in these elections?
Oh. Nevermind.
The NYTimes article focuses a lot on the results being a “surprise.”
The excepts in this post place less stress in the surprise element.
I fail to see how this result was surprising. From what I was hearing these results were pretty close to what was expected in the broad scale. Seems to me at this point you have to add 3 points to the Republican side of polling. If you fail to do that then the results might be mildly surprising but what would the argument for NOT doing that at this point? I’m a big believer in finding a plausible causal mechanism before assuming causality, but hasn’t the pattern been consistent enough since 2016 to assume the association? Is that period of time too short to make any such assumption? Can you just never do that if you can’t provide a plausible caial mechanism?
Another possibility is that people are too eager to find a narrative (e.g., Republicans did great, Dems did lousy) from weak and sparse data. Maybe the results of elections are just noisy around a fairly stable signal of roughly 50/50 outcomes?
Joshua:
The polls in Virginia were accurate. So you can’t just add 3 points.
Of course there’d still be some errors. But if you added 3 points to Republican poll numbers, would that improve your overall accuracy?
Joshua:
It depends on the year. In 2016 and 2020, yes. In other years, no.
In 2017-19, however, polls had essentially no partisan bias, and to the extent there was one, it was a very slight bias toward Republicans (0.3 percentage points). And that’s been the long-term pattern: Whatever bias there is in one batch of election polls doesn’t tend to persist from one cycle to the next. The Republican bias in the polls in 2011-12, for instance, which tended to underestimate then-President Obama’s re-election margins, was followed by two cycles of Democratic bias in 2013-14 and 2015-16, as previously mentioned. There is simply not much point in trying to guess the direction of poll bias ahead of time; if anything, it often seems to go against what the conventional wisdom expects. Instead, you should always be prepared for the possibility of systematic polling errors of several percentage points in either direction.
So maybe R’s overperformed polling 2012-2016, then 2020-2021?
https://fivethirtyeight.com/features/the-state-of-the-polls-2019/
“it makes sense that swing voters will want to balance that by voting [opposite] in the election that immediately follows.”
I don’t think that makes sense at all. Here’s what makes sense:
By definition swing voters are centrist compared to the parties. In most elections / districts (there are some exceptions) both parties need to play to centrist swing voters to win the election. But once parties get into government, they can drop the centrist lip service they used to get elected and revert to their preferred extremes, which then drives centrists to vote against them, not out of any desire for balance but because they don’t like the policies being advanced.
Grumpy:
That’s what I meant. The balance I’m talking about is policy balance, not just a balance of party labels. It’s complicated, though, for a few reasons:
1. Policies are multidimensional, and they don’t always fall on a clear left-right dimension.
2. A party can pursue a policy that is popular with voters, but centrist voters can still want balance if they think that this policy, which they like, is still pulling things too far to the left or the right.
3. Some contentious issues such as filibusters, redistricting, and vote counting are partisan and can have major consequences but are not policy issues in themselves. These relate to the idea of one party or the other having too much power.
I thought the emerging consensus was that the “swing” is primarily which voters turn out rather than individual voters actually switching parties between elections, and that the latter were pretty rare, and increasingly so. No?
Man it sure seems like the lurching around of the electorate (2020 vs 2020 GA runoff vs 2021) is clear: be less crazy.
Trump and the R ticket was too crazy, get tossed in 2020.
Republicans, who should have had an advantage in GA runoffs, implode ranting bamboo in the ballots, stolen Kraken gibberish, etc, get tossed.
Dems look to spend a gajillion dollars, botch the Afghanistan pullout, etc, get tossed.
Andrew, is there a missing latent variable called “platform bonkerness” that might explain summer of this?
The:
What you say is pretty much the theory that we have. It’s not so easy to tie it to particular issues, though.
I think there’s definitely “observability” issues at play here. In evaluating screw-ups (e.g., bad clock management at the end of an NFL game, tone-deaf commercials / campaigns such as Starbuck’s “let’s talk about race” 2015 SNAFU, etc.), I can’t help but think that a random dude/dudette on their couch in middle-of-nowhere Nebraska can take one look at something they only sort of understand and say “wow, that’s stupid. Don’t do that”.
The problem is how to properly observe this sort of person — once contacted / polled, the act of observation might _effect_ that unit. It would be great to know what “normal” people think, but they throw off orders of magnitude less data than those that are highly engaged (and are likely to be themselves more extreme). Look at how few people generate the overwhelming amount of Tweets!
Obviously, this is very hard to solve, but I demand action! Get cracking, Andrew!
The:
It can be easy to see that something is a screw-up but not be so clear on how to do better. For example, lots of Republicans didn’t like that Trump and his friends were denying the election outcome, but it was not clear how they could get from point A to point B without losing a bunch of voters. Biden getting bad news in the Afghanistan pullout wasn’t so good for him, but the alternative plan of just staying there didn’t work so great for Obama or Trump. Democrats passing trillion-dollar spending bills could alienate moderates, but their hope is to set a path for economic growth, which would help them in 2024. Etc. And all this is on the side of straight political calculation, even setting aside policy preferences. This is not to claim that politicians don’t make mistakes; my point is only that they’re busy trying to avoid mistakes but it’s not so easy in a competitive political environment.