← All posts

Why the margin of error has to be smaller than the margin of victory

Austria's 2016 presidential election was annulled not because fraud was proven, but because 77,000 mishandled ballots exceeded a 30,000-vote margin — and that arithmetic alone was enough.

It is just before 9 a.m. on May 22, 2016, and in a district electoral office somewhere in Austria, a sealed envelope is being opened ahead of schedule.

The envelope contains postal ballots — absentee votes cast by citizens who weren't voting in person. By law, those envelopes cannot be touched until the morning after the election. The district electoral board must be present. Witnesses must be there. None of that is happening. Someone decided, for reasons of convenience or haste or simple carelessness, to start early.

Across Austria that same day, variations of the same scene play out in multiple districts. Ballots counted before the permitted hour. Envelopes opened without the right people present. Votes tallied by unauthorized hands.

No one stuffed the ballot boxes. No one swapped the results. The Austrian Constitutional Court, when it examined the evidence months later, found no proof of actual fraud. What it found was something that turned out to matter just as much.

The arithmetic that ends an election

The final tally of the May 22 run-off showed Alexander Van der Bellen defeating Norbert Hofer by roughly 30,000 votes out of about 4.6 million cast — a margin of approximately 0.6 percent.

The court then did a different kind of arithmetic. It added up the votes affected by the procedural violations it had documented: postal ballots handled outside the law, in quantities that could be verified district by district. The total came to roughly 77,000.

Seventy-seven thousand against a margin of thirty thousand.

The court didn't need to prove that a single ballot had been altered. It only needed to prove — and this is the crucial insight — that the violations had occurred in sufficient numbers that if manipulation had taken place, it could have changed the result. The procedural rules weren't bureaucratic niceties. They were the mechanism by which the count could be independently verified. Once broken in numbers exceeding the margin, they could not be reconstructed. The result became, in a legal and democratic sense, unknowable.

On July 1, 2016, the Verfassungsgerichtshof annulled the entire run-off. A repeat election was scheduled for December.

This is a math problem, not just a legal one

Here is the principle the Austrian court articulated, translated into plain terms: your margin of error has to be smaller than your margin of victory, or you haven't actually decided anything.

This sounds almost too simple. But almost every contested election in living memory can be analyzed through this lens, and the analysis is almost always damning.

Think about what "margin of error" means in an election context. It isn't just counting mistakes. It is everything that creates uncertainty between the vote a citizen cast and the number that appears in the final tally: ballots processed outside legal procedures, machine miscounts that can't be independently verified, chain-of-custody breaks that leave a gap in the audit trail. When the total of all those uncertainties exceeds the winner's lead, you are not looking at a confirmed result. You are looking at a range.

And a range doesn't have a winner.

The Austrian case is almost perfectly designed as a teaching example because the numbers are unambiguous. 77,000 is larger than 30,000. The court didn't need sophisticated statistics. It needed arithmetic.

What a 'confirmed' hand count actually confirms

Now consider how you might try to resolve a problem like this. The obvious answer: recount the ballots by hand. Have humans look at every piece of paper. Correct any machine error.

This is appealing, and under the right conditions it's better than nothing. Georgia's 2020 presidential hand recount of roughly five million ballots — the largest manual tally in American history — affirmed the machine-tabulated outcome to within about a tenth of one percent. That is genuinely reassuring when the margin is measured in tens of thousands.

But look more carefully at what happened in Antrim County, Michigan, on a much smaller scale. After an error in the 2020 tabulation was caught and corrected, a full hand audit of about 15,700 presidential ballots was conducted. The hand count differed from the machine tabulation by about a dozen votes. Not because of fraud. Not because of a programming error. Because hand-counting is itself a method carried out by humans, and humans make mistakes — transposing numbers on a tally sheet, miscounting a stack, misreading an ambiguous mark.

A dozen votes in 15,700 is a small error rate. But here is the question the Antrim example forces you to ask: what if the race had been decided by eleven votes?

In that scenario, "the hand count confirmed it" is not a conclusion. It is a statement that one uncertain method was checked against another uncertain method, and they disagreed by more than the margin. You would be right back to a range without a winner.

This is not a theoretical problem. In 2006, Florida's 13th Congressional District was decided by 369 votes — a margin smaller than the irreducible noise in a large hand count of the same ballots. The machines were paperless, so there was nothing physical to recount at all. Eighteen thousand undervotes sat in the machines' memory, and the question of whether they represented voter choice or machine failure was permanently unanswerable.

"The hand count confirmed it" and "the Secretary of State says it's fine" are reassurances. They are not the same thing as proof you can check yourself.

Why 'no evidence of fraud' doesn't close the question

The Austrian ruling makes a point that gets lost in most election-integrity debates: the absence of proven fraud is not the same as the presence of a verifiable result.

The court was explicit. It didn't annul the election because it believed votes had been altered. It annulled it because the safeguards that would have proved they weren't altered had been violated in numbers too large to dismiss. The rules about who can handle ballots, and when, and under whose observation, aren't red tape. They are the audit trail. Break the audit trail and you don't have a result anyone can independently verify — you have a claim.

This distinction matters enormously in practice. "We found no evidence of fraud" is a statement about what investigators looked for and didn't find. It says nothing about what an attacker with access and motive could have done undetected. A result is trustworthy not when no fraud has been proven, but when the procedures make undetected fraud impossible to hide — because every step is observable by someone independent of the people running the count.

Colorado understood this when it pioneered the risk-limiting audit in 2017. The key insight behind an RLA is statistical precision: by tying the size of the hand sample to the margin of victory, you can state with defined confidence whether the announced winner is actually the true winner. In a race decided by 50 votes, you examine a very large fraction of ballots by hand. In a race decided by 50,000, you examine far fewer. The method scales the scrutiny to the uncertainty. It is the closest thing American election administration has to taking the Austrian court's arithmetic seriously.

But an RLA only works if there is a trustworthy paper record to audit against. Which brings you to the next problem.

The record you audit against has to be readable

In October 2020, a federal court in Atlanta was examining Georgia's Dominion ballot-marking system. The court found that the system tabulates votes from a QR code printed on the ballot — a code the voter cannot read. The human-readable text on the paper confirms the voter's choices. The QR code, which is what the scanner actually counts, could theoretically say something different. A voter has no way to verify they match.

This is a more subtle version of the same problem. You have a paper record. You can hand-count the paper. But what you are counting — the machine-readable barcode — is not independently verifiable by the human doing the counting without a device to decode it. The court noted, citing the National Academies, that no technical mechanism currently exists to guarantee that a vote-counting application produces accurate results and that testing alone cannot ensure systems haven't been compromised.

The lesson is uncomfortable: a paper trail is not automatically an auditable record. The artifact you count must be the same artifact the voter can inspect and confirm. If those two things are different, you haven't closed the verification loop — you've just added a layer of paper-shaped reassurance.

The German Federal Constitutional Court understood this in 2009, when it struck down electronic voting machines on the grounds that ordinary citizens must be able to verify the essential steps from ballot to result without specialist knowledge. Not auditors. Not experts. Citizens. The Netherlands reached the same conclusion independently, with its commission on the election process concluding that there are "no secrets in the election process" and that any method must be transparently checkable against a human-readable record.

The gap between announcement and verification

Here is what is still unresolved in most of the world's close elections, and why Austria 2016 is not a historical curiosity but a live warning.

When a result is announced, what you typically receive is: a number, a declaration, and an assurance. The assurance comes from the same institution that produced the number. Officials say the result has been verified. Auditors say the audit was completed. A Secretary of State issues a press release.

None of this is falsifiable by the public. You cannot, as a voter, go and check the underlying data yourself. You cannot pull up the precinct-level records, reconcile them against the reported totals, and confirm the arithmetic. In most jurisdictions you are, at some point, simply trusting that the people who ran the count did it correctly.

The Austria example shows exactly why that trust has a structural ceiling. The procedural violations there were documented — they were visible enough to bring to a constitutional court. But they were only documented because the losing party mounted a legal challenge, lawyers obtained records through litigation, and judges had power to compel disclosure. Most errors — and most potential manipulations — never face that kind of scrutiny, because no one who loses by 30,000 votes in a low-profile contest has the resources for a constitutional challenge.

The question the Austrian case poses is not "was this particular election stolen?" The court said it wasn't. The question is: in what fraction of close elections, at every level, do uncertainties exceed the margin — and who is in a position to know?

The answer, in most of the world, is: no one, because the data isn't public enough to check.

What 'independently checkable' actually has to mean

The fix is not more officials vouching for the process. It is not auditors who report to the same authorities that ran the election. It is not a press statement citing the absence of fraud.

It is precinct-level, machine-readable, publicly downloadable results — posted the moment each precinct's count is final, in a format anyone with a spreadsheet can reconcile against the certified totals. It is a paper record that shows the voter's choices in human-readable form and is scanned in a way that the scan is independently verifiable. It is an audit that is sized to the margin, run by people with no stake in the outcome, against ballots kept in a chain of custody that was never broken.

And crucially: it is a system designed so that the margin of procedural uncertainty — the ballots mishandled, the chain of custody broken, the QR codes unreadable — is structurally smaller than the smallest conceivable margin of victory. Not in most cases. Always.

That is what the Austrian court was really demanding. Not fraud-proofing. Verification-proofing. A system in which, when someone asks "how do I know the winner actually won?", the answer is not "trust us" — it is "here is the data, here are the records, here are the cryptographic proofs, and here is how you check them yourself."

Austria's 77,000 against 30,000 is not an embarrassing exception. It is the question every close election is silently asking, waiting for a system rigorous enough to answer it.

The shareable takeaway: A result no one can independently check isn't a confirmed result — it's an asserted one. If the number of votes that passed through a broken process exceeds the margin of victory, the math alone is enough to void the race. That's not a partisan claim. It's arithmetic. The only honest fix is a system where the public — not officials, not vendors, not auditors reporting to the people they're auditing — can verify the count themselves.

See how common this verification gap is across different countries and election systems →

Read the 2-minute version of why margin of error matters →

Sources

See all postsExplore the gaps