The following question came in the mail last week:
I am writing to seek your expert opinion on a peculiar observation regarding the recent election results in Iran. The total votes for the four candidates and the number of invalid votes are as follows:
Candidate A: 10,415,991
Candidate B: 9,473,298
Candidate C: 3,383,340
Candidate D: 206,397
Invalid votes: 1,056,159All these numbers are multiples of 3, which seems highly unusual. I would greatly appreciate your insight into whether this could indicate possible irregularities or if there might be a reasonable explanation for such a pattern.
I replied: This seems like the kind of thing that can just happen by chance. There’s nothing special about multiples of 3. I say this in general terms, with no specific knowledge of this election.
I didn’t think again about this until receiving a related email by someone who goes by the name Pedram McQubit:
In the last presidential election in Iran, the vote counts for all four candidates, as well as the number of invalid votes, were exactly divisible by three. This unusual pattern has raised concerns that the results may have been manipulated by the regime.
The suggested null hypothesis is that the regime did not manipulate the results. The alternative hypothesis is that the regime manipulated the results to inflate the appearance of support.
Some have argued that the probability of five random vote counts all being divisible by three is (1/3)^5, which is less than 0.5%. They believe this low probability justifies rejecting the null hypothesis. Others argue that, although this alone may not suffice to reject the null hypothesis, the regime’s lack of transparency and documented history of dishonesty provide further justification for suspecting manipulation.
The p-value calculated by some (so-called academics!), using a binomial test with a single trial, is stated as (1/3)^5. While I do not necessarily dispute the conclusion that the regime has manipulated the results, I find the methodology questionable, particularly the p-value calculation, the single trial consideration, and the apparent selective focus on this election without considering previous ones.
Given this context, I have several questions and would be grateful for your guidance. Even though the questions may seem basic, please feel free to provide answers as technical as you would like.
1. What hypothesis testing methods should be considered in such situations? Could a chi-squared test be applicable here?
2. With appropriate assumptions, do we have enough information to calculate a p-value and potentially reject the null hypothesis?
3. Are we justified in assuming manipulation by the regime, given its lack of transparency and history of dishonesty, based on this single rare event?
4. If a statistical approach can help us choose the most justified explanation, what would that justification be, and how can we arrive at it statistically and mathematically?
5. If a statistical approach is not suitable for this situation, what would be the reasons? For instance, could it be due to insufficient information? What alternative methodologies would you recommend?
I again replied that I don’t see any reason to think that the divisible-by-3 thing means anything.
How many elections have happened before we noticed one in which an event with likelihood of (1/243) happened?
There must be some name for this kind of fallacy.
Something unlikely happened => person calculates likelihood of event => person decides event could not have happened by chance.
I was born on January 1. (In a year with 366 days.) That was less likely than this particular shared divisibility.
Seems like the philosophical problem is that they want to use the same data to both generate the hypothesis and to test it.
I am interested to know what theory of vote tampering would result in divisibility by 3. The tests I’m aware of are based on digit frequency (Benford’s Law, for example), and have a sound philosophical basis.
Rick:
Also relevant is Bayesian reasoning: it’s hard to think of an alternative hypothesis in which vote counts would be more likely to be divisible by 3.
And, congratulations on your fun birthday! As you can see from the cover of Bayesian Data Analysis, third edition, it’s a statistically unusual day to be born.
I agree, without a theory/hypothesis that makes us expect results divisible by exactly three (or integers less than 10, or whatever), this amounts to suspecting any divisible by x outcome. In the current state, it is so vague as to explain almost any election result.
“Surprise” should be a function of multiple theories/hypotheses. But it is fine for hypothesis generation, if you suspect fraud then come up with a reason why division by three would be involved.
I can come up with an alternative hypothesis pretty easily… Someone generated the data randomly in Excel, didn’t know how to get it to work the way they wanted, so generated some numbers that were smaller than they wanted, then multiplied all of them by 3.
Not that I’m saying that’s what happened, it’s just an easy way to explain why they would *all* be multiples of 3.
In that case (someone doing who knows what with excel), it could have just as well been 5 though, right? All even numbers would amount to “whats the chance they are all divisible by 2?”
So still seems too vague without more assumptions about how the putative excel user generated the base random numbers.
Anon,
Yeah, true. I assume things like
1) people know how to generate random floats
2) people aren’t dumb enough to publish random floats as vote counts, and want integers
3) people might not be super smart about how to get random numbers that “look ok”
4) someone chooses some method to produce some that “look ok” except for the total size
5) round off (4) and multiply everything by 3 to get to plausible vote totals
I could see where people wouldn’t say they’d done (4) right unless the total size was within a factor of 3, so you’d expect (5) to be either “don’t multiply, or multiply by 2, or multiply by 3” but not “multiply by 4,5,6,7,13,28,etc” because instead they’d just redo (4) if the scale was way way off.
It’s easy to come up with plausible explanations, but they’re post hoc. There’s no real prior reason why I’d expect people to be multiplying things by 3. It just strikes me as **if someone were jimmying up numbers** then it wouldn’t be that implausible they’d need to make all the numbers bigger and just multiply what they’d jimmied up by 3.
I have heard of the “extreme value fallacy”, http://blog.thegrandlocus.com/2014/04/june-babies-and-bioinformatics, and to me it is the same fundamental issue as here. So you can post-hoc always identify some idiosyncratic characteristic of the vote tally that is low probability.
Andy W:
Yes, and on the very rare occasion that you can’t find such an idiosyncratic characteristic, that itself represents a statistically-significant analogy.
This is the statistical equivalent of the proof, well known to the kids in the math team, that all positive integers are interesting.
For the people who were not on the math team.
Assume x is the largest interesting number… This fact makes x+1 interesting but x+1 is larger than x contradicting the assumption that x is the largest interesting number, therefore there does not exist an x such that x is the largest interesting number.
proof by contradiction.
Pretty sure its this standard fallacy used in interpreting p-values:
https://rationalwiki.org/wiki/Confusion_of_the_inverse
There is an old article of Diaconis and Mosteller titled “Methods for Studying Coincidences” that addresses this sort of thing.
1/243 isn’t that rare given how many elections there are.
Nonetheless, the specific oddity of divisibility by 3 might be suspicious if there were a well known mechanism that could easily produce such a thing in the specific context of rigging elections.
The question “What hypothesis testing methods should be considered in such situations? Could a chi-squared test be applicable here?” reminds me of similar response I’ve heard on analogous questions. It’s a reminder that students can go through stats courses learning about a variety of tests without getting the logic behind them. Standard exercises don’t include “make a rough estimate of the number of how many post-hoc patterns with similar p-values might have caught your eye?”
I talked with someone who knew that some correlational stats on a causal question were heavily confounded and still wondered “shouldn’t they use chi-squared?”
Perhaps some of the new data-science courses will solve the “a little knowledge” problem by reducing the knowledge to zero.
Apologies for multiple copy/paste grammar errors.
I find it amusing that people are questioning whether an election in an autocratic regime well-known for the murder of its own civilians for dissent is fair because… the numbers are divisible by 3.
My assumption is that the election is a total sham, which is partly why I started giving plausible examples of how the multiple of 3 thing could come about, especially related to making the numbers up wholesale.
That’s fair enough – I just had to point it out!
I guess your point could be taken different ways.
I took it as: yes, of course it’s a sham. People that murder their own citizens:
a) probably cant figure out a better way to make fake election results than to make small numbers and multiply them up by some method similar to Daniel’s description;
b) probably don’t think too far ahead about whether other pepole will discover what they did, or maybe even really think it’s brilliant and no one will discover it;
c) don’t care that much if it is disocvered because they’ll deal with it the same way they deal with everything else: lies and bullshit and violence if necessary
I confess I agree with Daniel, I can’t quite get on board with Andrew’s shrugging it off as something that happens every day but we’re just all probability-dumb humans who are incapable of percieving it.
+10
Robin, etc.:
I take no stance on whether the vote totals were manipulated in some way. I just have no idea; I’m completely ignorant regarding the motivations, capacities, and inclinations of the Iranian election apparatus. If the vote totals were manipulated, I still think the numbers could still all be divisible by 3 just by chance.
To put it another way, if A is the event that those five totals are all divisible by 3, and B is the event that the vote totals were manipulated, then I’d assign Pr(B|~A) to be pretty much the same as Pr(B|A). So if you start with a prior probability Pr(B)=0.9, say, I’d have a posterior probability, just giving the above information, of Pr(B|A)=0.9.
I agree that it’s probably a sham election and that whether it’s divisible by 3 or any other small integer doesn’t much change the conclusion either way.
I think one should take into account the probability that all the votes are divisible by two (3.1%) , by five (0.032%) etc. Any of these conditions could seem ‘suspicious’
The possiblity that all the votes are divisible by some small number, ie the counts are ‘suspicious’, is about 3.5% which is too high to really be suspicious.
Yes, I was going to say the same. If 3 is suspicious then so would be many any other specific number that they are all divisible by, from 1 through 9, and then e.g., 10, 25, 50 and so on. Unless there is a specific reason that 3 in particular is or would be used for such things, we would also need to account for other numeric outcomes that might be deemed suspicious if they happened. It probably adds up to quite a lot.
[edit, obviously, not number 1 as implied by 1 through 9!]
What about the probability that the person fixing the election would try to cover his or her tracks by making sure the numbers had no common denominator?
There is of course the unquantifiable effect of the number of forking paths that led to the discovery of the divisibility by 3 on the amount of surprise.
Exactly. Generally it isn’t a valid application of a hypothesis test to see something that looks strange first and then find a test that tests the strangeness of this specific thing (even though there’s still some information in computing the probability 1/243 to see whether this is just a somewhat unusual thing or outright crazy (1/243 doesn’t have “outright crazy” format for me). Same with any Bayesian analysis that fixes model and prior after having seen the data.
Hat tip to whoever noticed they are all divisible by three. But I agree with the consensus here that that’s the sort of thing that happens if you choose a collection of 5 numbers. Not that unlikely that they will all be divisible by the same small prime, or have some other relationship.
But did anyone notice that four of the five are divisible by nine? Obviously this is not independent from the divisible-by-three thing — the ones that are divisible by nine are perforce divisible by three as well — but to me it takes this up a level from “meh, not all that unlikely, could be a coincidence” to “hmm, that does look kinda fishy.”
Hi
see these items …
1
https://jhanley.biostat.mcgill.ca/Reprints/HanleySLAS2018.pdf
in particular, the remarks re the case of the plague, and ‘researcher degrees of freedom, and why my colleague refused to do ‘after the fact’ probability calculations
2
https://jhanley.biostat.mcgill.ca/Reprints/LotteriesProbabilitiesHANLEY1984TeachingStatistics.pdf
See remarks re New Hampshire & Massachussetts lotteries.
pre-data, [which is when we can legitimately calculate probabilities] what does ‘same’ mean?
In your case, what, ‘BEFORE seeing the data’ would H_alt be?
why would it be divisible by ***3*** ” couldn’t it be by ***5*** ? ***6*** ? etc.. so, lots of possibilities of suspicious activity, all equally suspicious….
Newcomb mentions this subtlety in this piece..
https://jhanley.biostat.mcgill.ca/bios601/Intensity-Rate/Newcomb1860.pdf
but the difference b/w ‘(pre-‘) selected’ and ‘some’ [see p 18] is missed by a lot of readers
3
https://jhanley.biostat.mcgill.ca/Reprints/jumping_to_coincidences.pdf
if there was one word for it, might it be ‘multiplicity’ ?
If A = all vote totals divisible by 3, and B = the election was rigged, what is P(A | B)? I suspect it is very close to 1/243 as well.
Supposing the election was rigged, can Iranian authorities control the vote down to the last digit? It strikes me that if you’re going to rig an election, the most important thing to control is who wins. In the process of rigging the election, you would ideally like to leave as little evidence as possible that you’ve rigged it. Most elections have some process where subtotals from each of the vote counters are summed up to produce the final total. I suspect that if you looked at the spreadsheet that Iranian authorities made which sums up all of these totals, the math would be correct, because that’s the easiest thing to check about the election. If the election were rigged, it would leave less evidence to change the vote numbers before summing them up into the final tally, because you wouldn’t need the help of all of the vote counters. However, this would mean that you would no longer control the last digit. This is fine, though, because no one cares if the leading candidate got 10,415,991 votes or 10,415,992 votes. He won by a million votes. Being able to control the last digit of the vote totals is both not very useful, and leaves more evidence of rigging compared to controlling the overall winner.
For this anomaly to be the result of rigging rather than random chance requires a chain of odd behavior. It requires some person to generate fake vote totals that correspond to a 13% turnout rate. (This would be a weird choice – it’s much lower than the previous election, due to a boycott. The actual turnout of 40% in this election was embarrassing to the government.) It requires those totals to be tripled, rather than re-starting the generation process with better assumptions. Then it requires the infrastructure needed to rig the election that accurately.
For that reason, I don’t think this evidence should sway your mind one way or another about the election.