Skip to content
 

Pre-register post-election analyses?

David Randall writes:

I [Randall] have written an article on how we need to (in effect) pre-register the election—preregister the methods we will use to analyze the voting, with an eye to determining if there is voter fraud.

I have a horrible feeling we’re headed to civil war, and there’s nothing that can be done about it—but I thought that this might be a feather in the wind to prevent a predictable post-voting-day wrangle that could tip the country over the edge.

I wanted to get this out a month ago, and ideally in some less political venue. It got caught with an editor for several weeks, and now there is much less time to do anything. Still, better late than never.

Here’s what Randall writes in his article:

This November, on the morning after Election Day, Americans could awaken to find that both Trump and Biden are claiming to be the legitimate president-elect. . . . Passions are running so high that it will be hard to convince supporters of the loser to accept the result. . . . Since accusations of fraud seem almost inevitable, we should try to establish in advance a set of standards for determining whether or not it has occurred. . . .

Voter fraud can reveal itself in vote totals that differ markedly from the numbers that would be expected to emerge from a fair election. Statisticians can compare exit polls with recorded votes in each state and see if there are red flags. They can analyze precinct-level totals to look for “thumbprints” of voter fraud. It’s not as sure as a photograph of “mislaid” ballots dumped behind a post office, but it’s a very useful tool.

It’s not perfect, however. What if exit polling organizations don’t share their data? Even more seriously, what if partisan statisticians try out different statistical models until they come up with an analysis that “proves” or “disproves” voter fraud? If statisticians don’t say in advance what sorts of models they plan to use, they can cherry pick a method to prove whatever they like.

This sort of agenda-driven approach to statistics has plagued scientific research in recent years. It would make any post-election attempt to check on voter fraud unreliable too. America is seething with partisan rancor at the moment, and should that rancor explode, it could be catastrophic for the republic.

Statisticians and polling organizations should act before the election to outline the procedures they will use to assess whether voter fraud has taken place. Here’s what they need to do:

Set up born-open data collection that will allow everyone to see the raw data from exit polls.

Register polling organizations’ voting models in advance, so that everyone can understand why and how each exit poll differs from the final results.

Commit to presenting the results of all statistical models used to analyze election data, to eliminate the possibility of cherry-picking.

Draft and publicize a clear set of standard “best practices” criteria for post-election analysis.

These actions may require polling organizations to show outsiders their proprietary data and models. Perhaps access to the relevant data and models should be limited to a bipartisan group of statisticians who will sign legal agreements not to disclose the details of proprietary data and models. But Americans’ trust in “bipartisan experts” has never been lower. Given this fact, perhaps polling organizations should sacrifice their proprietary rights and make their data and methods public in the interest of the survival of the republic.

Open data and predefined statistical methods won’t be enough by themselves to save America from the toxic brew of partisan hatred. But they might make the difference that allows the country to accept the results of the November election in peace.

I have mixed feelings about these recommendations. I agree with the general point about openness, but realistically I don’t think people can really preregister their analyses. There are just too many different things to look at.

I think I’d have less faith in a set of preregistered analyses than in some sort of impartial commission of trusted experts. But it’s hard to imagine all the key parties involved setting up such a commission.

Look what happened in Florida in 2000: it wouldn’t have been so hard just to count all the votes in the state, but instead it became a political battle and the courts stopped the counting. It became a partisan matter rather than a simple vote count. Given that our political institutions couldn’t do it right in a relatively clean case of one state in 2000, I can well imagine any dispute in 2020 being so much worse. So, again, I’m sympathetic with Randall’s goals here, and I’m with him on the open data and analysis, not so much on the preregistration, though, given that this would limit what analyses could be done.

12 Comments

  1. Graham says:

    I find myself agreeing about the possibility of significant violence on/after the election, but I’m not sure that statistics will be of any use in curbing that violence. If people are really out in the streets with guns, and the President refuses to leave office, what good is a statistical analysis of the likelihood of voter fraud going to do?

    • MJM - WA says:

      Maybe we should instead have statisticians pre-register their priors so that we can identify an “impartial commission”? Based on the comment above, it would that Passions are running strong also amongst those who are statistically inclined.

    • confused says:

      Yeah… especially since I don’t think “fraud”, as such, is really the risk here.

      Something like 2000 is more likely to be problematic, in which the vote counting process is at issue, not the introduction of fraudulent votes. (In this year’s case, it might be about mail-ballot reception deadlines, etc.)

      However – the “base case” would likely be Biden winning strongly enough that no one-state flip could change the outcome, making a challenge a lot less plausible.

      For example, if Biden wins all Clinton states + WI MI PA AZ FL NC, that’s 333 electoral votes. In that scenario, there’s not much room for a plausible challenge: even flipping all three contested Midwest states (PA/WI/MI) *or* FL+AZ+NC wouldn’t change the outcome.

      And I think that’s closer to a “base case” than a 2000-style razor-thin-margin election.

      (And if Biden wins GA, OH, IA, ME-2, NE-2…)

  2. MichaelS says:

    Exit-Polls as a major, objective data source for statistical analysis ? No way.

    (what’s the typical margin of error in exit polls… and how would one determine that in a mass election where an unknown level of fraud was suspected ?)

    Also, a US national election is a very complex human/machine process…inherently loaded with error potential within >3,000 US counties using their own electoral rules & processes.

    A serious look at electoral & vote fraud is long overdue.
    There is no danger of civil war, but strong danger to America’s political establishment.

  3. Michael J says:

    Couldn’t you have both? Like there are probably a set of basic analyses that are a good idea to do and can be pre-registered. You can still do whatever analyses you want after the fact and give a justification for why it wasn’t reasonable to have pre-registered it. The consequence of course would be that the post-hoc analyses would be seen as less trustworthy (but not worthless) which is a benefit imo but I guess I can see why some people would view that as a negative.

    • Michael J says:

      Possible ideas for basic analyses would be something like a statistical calculation for how many ballots you would need to re-count in order to be X confident that the reported result is accurate.

  4. Rodney Sparapani says:

    Hi Andrew:

    This is really off the point. But, I’m sure that you just mis-typed this…
    “Look what happened in Florida in 2000: it wouldn’t have been so hard just to count all the votes in the state, but instead it became a political battle and the courts stopped the voting.”
    We both know that the courts did not stop the voting. They stopped the
    state-wide recount and there are legitimate arguments on both sides for
    that. There were recounts in some of the districts which were completed
    before the state of Florida’s supreme court’s action and
    the subsequent US supreme court’s reaction. Is that a better assessment?

    Let there be no mistake. I still enjoy your blog more than any other blog
    on the internet. It is how I start off my day for many years now!

    Rodney

  5. Fred says:

    Just want to point out a somewhat similar situation occurred in South Korea this April for its congressional election.
    Although Korea got over the biggest hump, COVID was obviously still around and contributed to a high numbers of early voting (27% compared to 12% four years ago).
    More extreme members of the conservative party spread fears about early voting being susceptible to fraud and that essentially suppressed its followers from voting early.
    While the exit polls showed that the ruling party would win, the early voting being skewed towards the ruling party made it a landslide victory, outside the margin of errors expected from the exit polls.
    This result worked as fuel for the fire to the fringe members, but thankfully, the conservative party and its leadership did stop short of fully embracing the election fraud conspiracy.

    Anyways, the main lesson is something most of the readers on this blog already know.
    Exit polls on the day of election is a biased sample of all voters and should be interpreted cautiously especially with a high level of early voting.

  6. JH11 says:

    {“…to analyze the voting, with an eye to determining if there is voter fraud.”

    Be careful with terminology.

    “Voter Fraud” is a subset of “Vote Fraud” which is a subset of “Electoral Fraud”.

    Malfeasance can occur anywhere in the very broad landscape registering voters, castng ballots, counting & validating cast ballots, reporting raw vote conts, officially consolidating and validating vote counts, and officially reporting election outcome publicly.

    We are really seeking anything that changes the “genuine” election results from the formal official results.
    Not so easy to do.

  7. confused says:

    The US is NOT headed for civil war. A civil war requires the military to split between the sides; no way that’s happening. In 1861 it was possible because loyalty was in many cases primarily to state rather than federal government – the US was much more “a collection of states” and much less “a single nation”.

    Increased civil unrest and local violence could happen, though. (But it seems quite unlikely. The most likely outcome seems to be a strong enough D victory that no single-state recount e.g. Florida in 2000 could change the outcome, so no opportunity for a plausible challenge to the outcome.)

Leave a Reply