State's Voting-Machine Certification Process
permission from an e-mail message written by Gilles Burger,
chairman of Maryland's Board of Elections)
interviewed Gilles Burger for a "CBS Sunday Morning" segment (air
date: 7/11/04), I was surprised at his confidence in electronic
voting machines, which he said had been used in three statewide
elections with great success. When I asked him how he could be
so sure that nobody had somehow rigged the machines -- after all,
they have no paper trail -- he sent me this description of
Maryland's testing and certification process.
with voting-industry jargon and acronyms, but each is spelled
out in its first use. --David Pogue
There are a battery of test and
certification activities conducted by different organizations
that our DRE system [Direct Recording Electronic voting machine]
undergoes as part of our acquisition process with Diebold:
1) Diebold first tests its system's compliance against its
design and requirements prior to release to the Independent
Testing Authority (ITA).
2) The ITA tests the system against the federal Voting Systems
Standards (2002 version).
3) The National Association of State Election Directors (NASED)
assigns a qualification number in response to a favorable
testing result from the ITA.
4) Maryland State Board of Elections (SBE) takes custody of the
system from the ITA (not Diebold) once NASED assigns the
5) SBE technical staff conducts two types of tests - a) Product
Tests to ensure the system components work according to expected
technical results, and b) User Acceptance Tests to ensure the
system functions according to expected functional results. (The
State Board of Elections doesn't pay Diebold until the
Acceptance Tests pass.)
6) The 5 SBE Board Members (which I'm one of) reviews the
results of the Product Tests and the User Acceptance Tests,
determines if the system meets specific Code of Maryland
Regulations standards, determines if the testing and
certification steps 1 - 5 were conducted properly, and then
grants the system a Certification for use in Maryland elections.
7) Once certified, an Independent Verification and Validation (IV&V)
organization installs the software/firmware to every touch
screen unit and GEMS server [the software that collects and
tallies the polling-place results], conducts installation and
checkout tests, and then controls the distribution of the units
to the respective Local Boards of Election (LBE) warehouses for
LBE custody. (Our IV&V vendor is completely independent of
Diebold or any other voting machine vendor.)
8) Once SBE finalizes the ballot designs for a specific election,
the LBEs load those ballot designs into the system and then
conduct Logic and Accuracy (L&A) tests - basically a series of
mock election scenarios to determine if results are as expected.
The scope of our L&A tests were significantly expanded over the
past year to test for trojan horses, time bombs, etc. Once these
tests are satisfactorily completed, the LBEs run a "zero report"
for each touch screen terminal, and then apply a unique numbered
wire seal to each touch screen terminal case. That number is
then recorded in an audit log.
9) For election day, the LBEs deliver each touch screen terminal
to their respective polling places and on election morning, each
polling place's 2 Chief Election Judges (1 Republican and 1
Democrat) break the numbered wire seals, record the number for
correlation, print out a new "zero report", and post those zero
reports for voter inspection.
10) (New for November's election) We are planning to conduct
parallel testing. We will preselect a number of touch screen
machines throughout the state and on election day we will
surprise the targeted polling places and remove the selected
machines from service. We will then conduct the L&A tests on
these machines to verify the machines operate as expected. We
are currently working out the details of this process.
11) The day after the election, we conduct 100% verification
that the results recorded on each machine's PCMCIA card matches
the election night unofficial tally.
I think [this list] pretty strongly addresses the worries about
vendor (or other nefarious) penetration. If one were to try to
alter the machines, it would be almost impossible to do so
without detection - this individual would not only have to hack
the machines without detection by any of these 11 testing and
certification steps, he would also have to guess what our
changeable ballot contents and formats would be in order to
successfully fool the system without detection.