← All posts

What separates a real audit from a rubber stamp

Every election gets an "audit." Almost none of them can actually change a result — and the word is doing a lot of work to hide that.

It is the morning of November 4, 2020, and a clerk's office in Windham, New Hampshire is staring at a problem. The ballots have been counted. The machine tapes have printed. Four Republican candidates for State Representative have won their seats — but by margins thinner than anyone expected. One candidate, Kristi St. Laurent, lost to the fourth-place Republican by only 24 votes.

Her campaign requests a hand recount.

The hand count comes back. St. Laurent still loses — but the numbers have moved. All four Republicans gained roughly 300 votes each. Every Democrat in the race lost roughly 99 votes. The machine count and the hand count have diverged by hundreds in a small-town race. The discrepancy is not subtle. It is not a rounding error.

It is a smoking gun that requires an explanation.

What a real audit looks like

New Hampshire did something that most places would not do. Under SB 43, the state legislature ordered a full forensic audit — not a recount, not a press release, not a reassuring statement from the Secretary of State's office. A forensic audit, run by independent experts: Harri Hursti, Mark Lindeman, and Philip Stark, three of the most credentialed election-security researchers in the country.

They got access to the machines. They got access to the ballots. They ran experiments.

What they found was this: a folding machine the town had leased to prepare absentee ballots did not fold the paper along the printed score lines. The folds ran directly through the oval vote targets. The AccuVote optical scanners — the same family of machines studied by Princeton researchers in 2006, when Ariel Feldman, J. Alex Halderman, and Edward Felten demonstrated how a minute of physical access could compromise an entire system — read a substantial fraction of those fold creases as marked ovals. The team estimated about 44 percent of folds through vote targets were being interpreted as votes. Experimental runs produced rates anywhere from roughly 20 percent to over 72 percent. White powder had also built up inside the scanners, obstructing the lenses.

No malware. No fraud. No tampering. Just a crease in the wrong place, and hundreds of votes silently miscounted.

The report concluded the election was, for the most part, well run. Read that sentence slowly. A well-run election, in a functioning democracy, by competent officials, produced a miscount of hundreds of votes in a race decided by 24 — and nobody would have known without an independent, binding, forensic inquiry that actually examined the physical evidence.

That is what a real audit looks like. It is binding (the legislature ordered it), independent (the team had no relationship with the officials who ran the count), and evidence-based (they examined paper, machines, and test runs, not just official certifications). It produced findings that could — and did — change the factual record of what happened.

The full SB 43 forensic audit report is public. You can read every methodology decision, every experimental result, every caveat. That transparency is not incidental — it is the whole point.

What most "audits" actually are

Here is the uncomfortable truth that officials rarely volunteer: in most U.S. jurisdictions, and in many democracies around the world, what is called an "audit" after an election is a non-binding spot check that cannot, by design, change the certified result.

A typical post-election audit might check 1 percent of ballots. Or 5 percent. Or a fixed sample across precincts. The sample is usually chosen by the county or state running it. The people doing the checking are usually county or state employees. The finding — almost invariably — is that the machine count looks about right, within expected tolerances.

And when the audit is done, the result stands regardless of what it found.

This is not an audit in the forensic sense. It is a confidence ritual. It is a procedure designed to produce reassurance, not to challenge authority. The word "audit" does the rhetorical work of conveying rigor without legally requiring any.

An audit that cannot change the result is not an audit. It is a press release with extra steps.

Think about what "non-binding" means in practice. An official spot-checks 1 percent of ballots. Everything looks fine. The election is certified. Later, someone finds a problem — a fold through a vote target, a misconfigured tabulator card like the one in Antrim County, Michigan in 2020, where wrong unofficial results were published because a clerk had reprogrammed some but not all tabulator cards after a last-minute race change. In Antrim, the error was caught not by any audit, but because the result was so implausible for a reliably Republican county that it triggered immediate scrutiny. A subtler error in a closer race, in a county nobody had strong priors about, might not have triggered anything.

The hand audit that eventually checked every presidential ballot in Antrim differed from the machine count by about a dozen votes out of roughly 15,700 cast. Officials called this confirmation. But in a race decided by a handful of votes, a method that is itself off by a handful of votes cannot actually settle the question. "The audit confirmed it" is a reassurance, not a proof.

What a risk-limiting audit actually promises — and what it requires

Colorado did something genuinely different. In November 2017, following its general election, it became the first state in the United States to complete a statewide risk-limiting audit (RLA). The Colorado Secretary of State announced the completion on November 22, 2017.

An RLA is not a fixed-percentage spot check. It is a statistical procedure that answers a specific question: given the reported margin, what is the probability that the reported winner actually lost? The audit draws ballots at random and compares them hand-to-machine until either the audit reaches a pre-set confidence threshold, or the sample grows so large that the original result should be overturned.

The critical feature is that the sample size scales with the margin. A race decided by 50 votes requires checking many more ballots than a race decided by 50,000. Colorado started with a risk limit of 9 percent — meaning no more than a 9 percent chance of certifying a wrong winner — and later tightened it to 3 percent. The Colorado Secretary of State's RLA FAQ explains the methodology in plain language.

This is meaningfully different from a non-binding spot check. An RLA has a defined statistical guarantee. If the audit triggers a full hand count, a full hand count happens. The math is public. Anyone can check the confidence calculation against the ballots sampled.

But an RLA has one non-negotiable prerequisite: a trustworthy paper record to audit against.

This is where Sarasota County, Florida becomes relevant. In 2006, roughly 18,000 undervotes appeared in a Congressional race decided by 369 votes. The county used paperless DRE touchscreen machines. The GAO tested the systems and found no machine malfunction but noted, plainly, that a voter-verified paper trail could have provided independent confirmation. No paper record existed. There was literally nothing independent to audit against. A risk-limiting audit in Sarasota 2006 would have been mathematically impossible, because the evidence layer the math depends on did not exist.

You cannot run a real audit on a system that leaves no independently checkable artifact. The paper ballot is not a quaint backup. It is the audit's raw material.

What Georgia 2020 proved — and what it left open

Georgia's 2020 post-election audit is frequently cited as proof that the system works. It deserves a closer look than it usually gets.

After the November 2020 presidential election, Georgia conducted a full hand tally of all approximately 5 million presidential ballots — 159 counties, 41,881 batches, completed in under six days. The hand count affirmed the machine-tabulated outcome, with a variation between the original count and the hand tally of about a tenth of a percent.

Credit where it is due: that is a genuinely impressive feat of election administration, and the existence of voter-marked paper ballots made it possible. Georgia Public Broadcasting and NPR both covered it.

But here is what the Georgia audit could not do: it could not verify that the ballots it counted were the same as the ballots voters intended to cast. Georgia uses Dominion ballot-marking devices (BMDs). A voter touches a screen; the machine prints a ballot with human-readable text and a QR barcode. The tabulator reads the QR code. In Curling v. Raffensperger, a federal court in October 2020 quoted expert testimony finding the system "does not provide a verifiable and auditable ballot record because it relies on the QR code for vote tabulation and that code itself cannot be read and verified by the voter." The court also cited the National Academies' 2018 conclusion that no technical mechanism currently exists to ensure a vote-counting application produces accurate results.

A hand count audits what the QR code says. It does not — cannot — audit whether the QR code matches what the voter touched on the screen. If the BMD software misrecorded the voter's choice before printing, the paper would look fine to a hand-counter and the miscount would pass every audit.

This is not an accusation. It is a logical constraint. A real audit requires that the artifact being counted is the same artifact the voter verified. When those two things are different — when the machine reads a barcode the voter cannot read — the audit is checking the machine's output, not the voter's intent.

The gap nobody names out loud

So we have a spectrum.

At one end: Windham's forensic audit. Independent experts, binding mandate, physical evidence examined, experiments run, findings that diverged from official counts and changed the public record of what happened. Expensive, slow, politically uncomfortable, and the only reason we know what actually went wrong.

Slightly up the spectrum: Colorado's RLA. Mathematically grounded, statistically honest about confidence levels, scalable to the margin. Still depends on auditable paper, still requires the paper to reflect the voter's actual intent, but far more rigorous than a fixed-percentage spot check.

At the other end: the standard post-election audit in most places. Non-binding. Run by the same officials whose count it is supposedly checking. Fixed percentage, often tiny. Unable to trigger a result change. Designed, in practice, to produce reassurance.

Most elections live at the wrong end of that spectrum. And the word "audit" floats across all of them equally, doing reputational work it has not earned.

The German Federal Constitutional Court said it plainly in its March 2009 ruling banning voting computers: the essential steps of voting and counting must be examinable by the citizen without specialist knowledge. Not examinable by officials. Not examinable by vendors. By citizens. That is the constitutional standard a democracy owes its voters.

The Dutch government's 2007 commission on elections, "Stemmen met vertrouwen" — Voting with Confidence — reached the same conclusion from a different direction: transparency and checkability are not optional features of a trustworthy count. They are what makes a count trustworthy at all.

Neither of those conclusions is radical. Both follow from the same simple premise: a result no one can independently verify is not a verified result. It is an announced result backed by trust in whoever announced it.

Trust in officials is not the same thing as proof. In Venezuela in 2017, Smartmatic — the company that had provided Venezuela's automated voting system since 2004 — publicly stated it knew "without any doubt" that the announced turnout had been manipulated, estimating the difference between actual and announced participation at at least one million votes. The company that built the system disowned the result. There was no independent audit to appeal to. There was nothing.

That is the terminal case. But the gap between that and the Windham forensic audit is a spectrum, not a bright line, and most elections sit somewhere in the middle — with procedures called "audits" that function more like Venezuela's than like Windham's.

What would make it checkable by anyone

The fix is not to trust better officials. It is to design systems that do not require that trust.

A genuine post-election audit has three properties. It is independent — conducted by people with no stake in the outcome and no reporting relationship to the officials who ran the election. It is binding — capable of producing a different certified result if the evidence warrants it. And it is evidence-based — it checks the physical or cryptographic record against the machine count, using a method whose error rate is known and published.

Windham had all three, after the fact, because a legislated order forced it. Colorado's RLA has the statistical rigor and the binding force. Neither is sufficient on its own if the underlying record — the ballot — cannot be independently verified to reflect the voter's intent.

The harder conclusion is this: even hand counts carry irreducible human error. The Antrim County hand audit differed from the machine tally by about a dozen votes. The Georgia hand tally differed from the machine tally by about a tenth of a percent across 5 million ballots. "A hand count confirmed it" does not mean zero error. In a race decided by twelve votes, a method that is itself off by twelve votes cannot definitively settle the question. In a race decided by 369 votes — like Sarasota 2006 — the absence of any paper record means the question cannot even be asked.

What would actually make results checkable by anyone is a combination of things that currently exist in pieces but rarely together: voter-marked paper ballots whose marks unambiguously reflect the voter's choice; routine, binding, independent audits with a published statistical confidence level; precinct-level results published in machine-readable formats the moment they are final; and full chain-of-custody documentation that is itself public and auditable.

None of those things requires trusting any particular official, vendor, or party. That is the point. Verifiability means the public can check, not that officials have checked and told you the result.

Right now, in most places, the word "audit" means the latter. Windham showed what the former looks like — after the fact, under pressure, because someone counted by hand and noticed the numbers didn't match.

That should not be how we find out.

See how the non-binding audit gap shows up across jurisdictions | Explore the full case atlas | Read the 2-minute version of this argument

Sources

See all postsExplore the gaps