E-voting

Evaluating Electronic Voting

Systems Equipped with
Voter-Verified Paper Records
To increase public confidence, states are increasingly
considering electronic voting systems that provide voter-
verified paper records. An analysis and evaluation of
New Jersey’s criteria against several different e-voting
machine types revealed potential threats—and possible
solutions—on privacy, security, and performance issues.

G
NIRWAN overnments around the world are increas- must carefully
ANSARI, ingly replacing traditional paper ballots evaluate a print-
PITIPATANA and punch cards with electronic vot- er’s performance and its integration with the overall
SAKARINDR, ing systems such as the now 20-year-old voting system.
EHSAN direct-record electronic (DRE) system.1 In such Federal and state election commissions have made
HAGHANI, electronic voting (e-voting) systems, vendor-specific different recommendations for evaluating and cer-
CHAO ZHANG, software controls system functionality, records and tifying e-voting systems.4-8 In the US, states have
ARIDAMAN K. counts electronic votes, and generates the final re- developed different requirements that target their
JAIN, AND YUN sults. Although e-voting systems are subject to fed- particular needs. The Attorney General’s Office of
Q. SHI eral and state certification, election officials and New Jersey issued criteria for e-voting machines
New Jersey public advocates have raised many questions about equipped with printers4 and asked the New Jersey
Institute of their privacy, security, and performance. Before cer- Institute of Technology to test the various systems
Technology tifying a system, election officials must evaluate its against these criteria.9-12 As we discuss here, in the
hardware and software performance and its source testing and analysis process, we encountered several
code. However, simple lab testing is inadequate for issues of concern and formulated recommendations
detecting errors at all levels,1,2 and undetected flaws for addressing some of them.
can have devastating consequences in elections.
Furthermore, weak requirements are fundamentally System requirements
inefficient for ensuring voting system performance A DRE voting machine with VVPRS capability in-
and security. 3 Thus, each state must examine and cludes a ballot display unit, a voting panel, internal
evaluate its own requirements to ensure the vot- and external memories, a printer, a paper-record dis-
ing system’s functionality, security, durability, and play unit, and a paper-record storage unit. The voting
accessibility,4 as well as the voting results’ secrecy, systems we tested all use thermal printers and adopt
privacy, and accuracy. one of two methods: In cut-and-drop VVPRS, the in-
To increase voter confidence, some states have dividual printed paper records are cut and dropped
proposed—and in some cases, mandated—the ad- into a storage unit; in continuous spool VVPRS, the
dition of printers to voting machines.4-5 This lets vote selections are printed on a paper roll that rolls
voters verify their voting selections on paper re- continuously from one spool to another.
cords; officials then couple the electronic record of As Figure 1 shows, New Jersey’s criteria define the
each vote with a printed paper record. Using DREs system’s component functionalities.
with voter-verified paper-record systems (VVPRSs)
should instill full public confidence in the electoral Privacy requirements
process. To certify such a system, however, analysts Voter privacy requirements are as follows:

30 PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/08/$25.00 © 2008 IEEE ■ IEEE SECURITY & PRIVACY
 E-voting

• Voters must be able to privately

and independently select candi-
dates on the DRE machine and
verify their selections on the
printed paper record.
• The voter’s identity can’t be re-
corded or identified in the elec-
tronic and paper records, in the
method that creates and stores
these records, or in the method
linking the DRE electronic-bal-
lot-image record to the corre-
sponding paper record.
• These same privacy protections
must exist for visually impaired
voters using audio-assistance.

Further, the electronic and pa-

per records must be created, stored,
and audited in a way that preserves
their privacy.

Security requirements
Security requirements exist for the
DRE System, the VVPRS, and for
the vote records.

DRE system security. The vot-

ing system must prevent tamper-
ing with the election, the voting
results, and the system’s func-
tionality. The voting system must
withstand power-source failure
and use a reserve battery to avoid
poll closure. The DRE must detect
any error or malfunction, report it
to the election official and to the
voter, and correctly record any Figure 1. Evaluation criteria derived from the New Jersey Attorney General’s Office criteria for
such incident in the internal audit direct-record electronic (DRE) machines with voter-verified paper record systems (VVPRS).
log. Finally, the US cryptographic The requirements in red italics weren’t part of the testing team’s contract and therefore
module validation program must weren’t tested.
test and approve all voting system
cryptographic software.4
Vote record security. The voting systems can protect
VVPRS system security. The VVPRS must draw pow- the vote record (that is, the paper and electronic re-
er from the DRE system or from the same circuit the cords) by using digital signatures, which identify and
DRE system uses to draw its power. Security features authenticate the vote records, and error-correcting
must be available for maintaining VVPRS integrity; for codes, which can help detect barcode read errors. The
example, VVPRS printer components and the DRE- vote record should be fully recoverable in the event of
VVPRS connections must be secure. The VVPRS must malfunction or tampering.
be able to detect any error or malfunction and report it
to the election official and the voter. The VVPRS must Verification requirements
not be able to communicate externally to any system Before casting a vote, voters must be able to review
other than the DRE system. If any supply replacement and verify their selections on the DRE system and the
is required, it must not circumvent the security features corresponding paper records. New Jersey’s criteria let
that protect the paper records’ secrecy. voters reject and recast the ballot up to two additional

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 31

 E-voting

times. Official workers must also have the opportu- Testing techniques
nity to compare the electronic and paper records after We designed and conducted four testing approaches—a
the election for audit and recount purposes. There- single test, a 1,200-vote simulated test, a 14-hour test,
fore, the electronic and paper records must be linked and a 52-vote test—to examine VVPRS against certain
through a unique identifier. state requirements; for all, we used accepted scientific
practices and methodologies. We recruited students
Integrity requirements with different backgrounds to act as “mock voters”;
There are separate integrity requirements for the paper they ranged from undergraduates to PhD candidates.
and electronic records. The paper record must include Mock voters cast votes in various voting scenarios,
every contest the voter casts on the DRE system, in- each of which represented particular selections of an
cluding write-ins and undervotes. It must also iden- election’s contest positions. We printed the scenarios on
tify the election, the election district, and the voting cards, which the testing team shuffled to achieve ran-
machine. Moreover, the paper record’s contents must domization prior to giving them to the mock voters.
be machine readable (using barcodes, for example) in Each voter made selections as indicated on each sce-
case a recount and audit is needed. As noted earlier, nario card under the testing team’s close supervision.
the paper record must contain error-correcting codes
to detect read errors. Finally, election officials must be Ballots
able to distinguish between an accepted and rejected We adopted two ballot types: one long, one short. As
paper record. Figure 2 shows, the long ballot—which we used for the
Electronic records must include all votes cast by 14-hour and 52-vote tests—contained 19 items to vote
the voter, including write-ins and undervotes. The on. We designed 12 voting scenarios to represent all pos-
electronic record can include some security identities, sible choices, including eight party voting scenarios that
such as digital signatures for an individual record and were completely balanced (two parties for seven contests;
for the election’s entire set of electronic records. seven “yes” or “no” questions; and 10 candidates listed
for the charter study commission). In the eight voting
Format, design, scenarios, each position got four Democratic (D) and
and functionality requirements four Republican (R) candidate votes, and each question
Developers must create a voting machine that works got four “yes” votes and four “no” votes. We also had
with minimum disruption on Election Day. The ma- four supplementary voting scenarios that we designed
chine must be provisioned with a sufficient amount to test possibilities not included in the eight scenarios.
of voting supplies, such as paper and ink. If the DRE Finally, we considered two additional cases from among
runs low on paper, it must let the current voter fully the 12 scenarios to test whether voters could reject and
verify all of his or her vote selections and successfully recast their ballot during the 14-hour test. In the first
cast the ballot without disruption before a paper re- case, voters voided their first set of selections (one of the
placement. Developers must design the VVPRS to 12 scenarios) and recast their votes for the second set
function only as a printer; it must not be capable of (another of the 12 scenarios). In the second case, voters
networking or communicating externally with any voided their first two sets of selections and recast their
system other than the DRE system. Finally, the elec- votes for the third and final selection.
tronic-ballot-image record’s format must be publicly We used a short ballot in the 1,200-vote test; this
available and non-proprietary; in addition, the ballot’s ballot featured the same 12 voting scenarios as the long
barcode must use an industry standard format and be ballot, but omitted the charter study commission and
readable with any commercial barcode reader. had few questions. The ballot contained eight party
voting scenarios (again, completely balanced, with two
Documentation and parties for five positions, and “yes” or “no” votes for four
certification requirements questions) and four supplementary voting scenarios.
The vendor must supply documentation for election For all volume tests, we retained summaries of the
worker training, voting instructions, and malfunction following records:
recovery. The vendor must submit all federally cer-
tified Independent Testing Authority reports on the • tabulation of final paper records,
DRE with VVPRS. • tabulation of the final paper records’ scanned bar-
codes,
Examination requirements • electronic records, and
The VVPRS must be subject to the State Voting Ma- • the closed poll’s tally report.
chine Examination Committee’s scrutiny. In addition,
the vendor must provide the state with the DRE and Each summary gave the vote counts for each contest
VVPRS source code for independent testing. candidate (including the questions).

32 IEEE SECURITY & PRIVACY ■ MAY/JUNE 2008

 E-voting

LONG BALLOT MAJOR PARTY SCENARIO NUMBER SUPPLEMENTARY SCENARIO NUMBER

1 2 3 4 5 6 7 8 9 10 11 12
Position PRES R D D D D R R R PET1 PET2 WRITE-IN
US-S D R D D R R R D WRITE-IN WRITE-IN
US-H R R D R D R D D WRITE-IN WRITE-IN
F 3-YR-1 R R D D R D D R WRITE-IN WRITE-IN
F 3-YR-2 D R D R D D R R WRITE-IN WRITE-IN
F 2-YR R D D R R D R D WRITE-IN WRITE-IN
TOWNSHIP D D D R R R D R WRITE-IN WRITE-IN

Question 1 NO YES NO NO YES YES NO YES

2 NO NO YES NO NO YES YES YES
3 NO YES NO YES NO YES YES NO
4 NO YES YES NO YES NO YES NO
5 NO NO NO YES YES NO YES YES
6 NO YES YES YES NO NO NO YES
7 NO NO YES YES YES YES NO NO

Charter study 1 N1 N6 N1 N4 N10 N6 N8 N1 N6 N9

commission

2 N2 N7 N2 N5 W1 N7 N9 N2 N7 N10
3 N3 N8 N3 W2 W3 N10 N3 N8
4 N4 N9 N4
5 N5 N5
No. of 5 4 3 2 3 0 3 3 5 3 2 0
charter study
commission
voted

Figure 2. The long ballot. The 12 voting scenarios—eight major party and four supplementary—represent all possible choices. “R” and
“D” stand for a vote for a Republican or Democratic name, respectively. A blank space indicates a “no” vote for that position. For the
charter study commission, N1, N2, …, N10 indicate a vote for Name1, Name2,…, Name 10, respectively. W1, W2, and W3 are the three
write-in names for the charter study commission.

Single test to two-hour time slot using the long ballot. We gave
In the single test, we ran a one-time examination each mock voter a set of shuffled scenario cards derived
of specific criteria using different testing methods. from eight sets of eight major party voting scenarios
For example, the test might be a physical inspection and one set of four supplementary voting scenarios.
of various DRE and VVPRS components. In many We also randomly inserted questionnaire cards that
cases, we retrieved, studied, and compared paper asked voters questions about the voting scenario.
records, electronic records, and barcodes. For ex-
ample, we verified deployment of error-correction 1,200-vote simulated test
codes and digital signatures by closely examining The state’s criteria recommends that each machine be
these records. In some cases, we forced incidental capable of processing 750 ballots; we designed this test
and procedural hindrances—such as a paper jam—to to investigate the voting system’s response to a larger
observe the effect. We also closely examined all ven- than expected number of ballots, which tend to over-
dor documentations. load the system’s capability. Using a short ballot and
a scripted program, we ran a simulated test in which
14-hour test each machine continuously generated 1,200 votes. To
Our 14-hour test emulated actual physical voting sit- reach the 1,200 vote total, the test generated each of
uations over a 14-hour period (representing an entire the eight party-voting scenarios 125 times, and each
election day). Each mock voter cast votes over a one- of the four supplementary voting scenarios 50 times.

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 33

 E-voting

Table 1. Configurations of the four machine types.

COMPONENTS MACHINE TYPE 1 MACHINE TYPE 2 MACHINE TYPE 3 MACHINE TYPE 4

DRE SYSTEM
User interface 32” LCD touch screen A panel with matrix- 15” LCD touch screen 15” LCD touch screen
liked switch buttons
and a write-in LCD
screen and keyboard

Ballot-activation An RFID smart card with a card An activation button An RFID smart card An Infrared Data
device encoder on an official panel with a card encoder Association (IrDA)
and a button at the proprietary device with
back of the DRE an encoding cradle

Electronic-record Built-in memory, flash drive, DVD Built-in memories, Built-in memories, a Three built-
storage device a proprietary device proprietary PCMCIA in memories; a
designed by Personal device compact flash card;
Computer Memory a proprietary, IrDA-
Card International designed device
Association (PCMCIA)
Audio-assisted voting A modified keyboard with four A proprietary four- A proprietary four- Four buttons with
interface different button shapes and a button panel and a button panel and a different shapes on the
headset headset headset DRE and a headset

Magnification Yes Yes Yes Yes

device?
Externally connected No Yes (an additional No Yes (an additional
printer? printer and a VVPRS printer and a VVPRS
printer simultaneously printer connected to
connected to the DRE) the DRE one at a time)
Power source Primarily alternating current (AC), Primarily AC, with a Primarily AC, with a Primarily AC, with a
with a battery backup (up to 16 battery backup (up to battery backup (up to battery backup (up to
hours, with a maximum of four 16 hours) two hours) two hours)
battery packs)
Power daisy chains Yes (three AC outputs) Yes (10 AC outputs) Yes (one AC output) Yes (six AC outputs)

Interfaces/adapters 2 Personal System/2 ports (PS/2 1 IEEE-1284; 1 RJ-45; 1 IEEE-1284; 1 RJ-45; 1 RS-232; 1 IrDA slot;
(observed) to USB adapter also provided); 2 PCMCIA slots 2 PCMCIA slots 1 compact Flash slot;
4 USB ports; 1 IEEE 1284; 2 audio
recommended standard ports (RS-
232); 1 supervideo graphics array
(SVGA); 1 registered jack (RJ-45); 1
Ethernet; 1 RFID slot; audio
VVPRS COMPONENT
Printer type Cut-and-drop Cut-and-drop Continuous spool Continuous spool

Paper type Thermal Thermal Thermal Thermal

Paper size (print 80 mm 80 mm 80 mm 80 mm
width)
Paper-record storage Metal box Bag Spool Spool
unit
Paper supply 600 votes 500 votes 120 votes 120 votes
capacity
(approximate)

34 IEEE SECURITY & PRIVACY ■ MAY/JUNE 2008

 E-voting

In cases in which the machines lacked the script capa- displayed on the DRE screen. Consequently, they’d
bility to automate this test, we had mock voters cast likely seek assistance from a poll worker, who might
the 1,200 votes manually. see vote selections (displayed on the DRE screen and
in the paper-record display unit), thus violating visu-
52-vote test ally impaired voters’ privacy.
Finally, we designed a 52-vote test to investigate the
special case in which the paper record extends to mul- Record privacy
tiple pages. This criteria applies only to VVPRSs us- We found several violations of the state’s paper and
ing the cut-and-drop method (in this case, machine electronic record privacy requirements (Figure 1, sub-
types 1 and 2). We ran this test using the long ballot category 1.2). First, regarding the creation and store
and mock voters. requirements, the type 4 machine recorded the elec-
tronic record when voters approved their ballots on
Problems and criteria exceptions the DRE screen, rather than after they approved the
We tested four machine types with different configu- paper record.
rations from different manufacturers. Table 1 shows Second, regarding the linkage between paper and
the machines’ features; to maintain confidentiality, electronic records, the machines did use a unique
we don’t disclose the vendors’ identities, the machines’ identifier to link the two records. However, in ma-
models and serial numbers, or other proprietary in- chine types 3 and 4, the reconciliation process was
formation. As Table 2 summarizes, the systems didn’t time consuming and difficult given a large vote vol-
comply with all of the state’s criteria during our tests. ume. With the type 2 machine, it would likely be im-
The problems and criteria exceptions fell into several possible to reconcile the two records if one or more
general categories as follows. paper records were lost.

Voter privacy DRE security mechanisms

We found several violations of the state’s require- We encountered several problems with DRE secu-
ments for voter privacy (Figure 1, subcategory 1.1). rity mechanisms (Figure 1, subcategory 2.1). Regard-
First, regarding vote casting, reviewing, and verify- ing error detection and notification, machine type 1’s
ing, machine types 2, 3, and 4 offered physical enclo- DRE didn’t suspend voting when the printer cable
sures—including two-sided panels, a top panel, and was disconnected. It also failed to emit any signal to
a curtain—but failed to provide full voter privacy. the election official. Although the DRE recorded the
Observers placed around the voting machine were vote electronically, the VVPRS didn’t print the pa-
able to read vote selections on the DRE screen and on per record. Machine type 2’s DRE displayed an error
the paper record in its display unit. Although sneak- notification when a mechanical error or malfunction
ing inside or loitering around poll booths is illegal in occurred, but the error message didn’t always reflect
real elections, these machine types still pose a privacy the actual problem. Also, the DREs of types 3 and 4
threat to voters. To mitigate such a threat, these ma- couldn’t detect a paper jam. Regarding internal audit
chines must be strategically placed, such as facing out- log criteria, when the printer cable between the DRE
ward from a wall. and VVPRS was disconnected, the type 2 and 3 ma-
Second, the machines had problems regarding chines’ DREs didn’t record the incident.
protecting links between votes and voter identities.
One machine (type 4) printed the paper record with VVPRS security mechanisms
the exact voting date and time. Comparing this time- We found three violations in this category (Figure 1,
stamp to the poll log (which records the voter’s check- subcategory 2.2). First, regarding supply replacement
in time) could match the paper record to the voter and security, when we restocked the paper supply, type 3
thus reveal his or her identity. and 4 machines didn’t have sufficient security locks to
Finally, we encountered privacy issues when we protect the paper records in the paper-record storage
tested audio-assisted voting on a type 2 machine. In unit. More important, in machine type 1, there was
this case, the cut-and-drop printer printed the paper a slit in the paper-record storage unit, through which
record in multiple pages. After printing the first pa- corrupt election officials could slide fraudulent paper
per-record page, the system displayed a message on the records after they’d unlocked the printer cover to re-
DRE screen rather than providing an audio message. stock the paper supply.
The displayed message prompted the voter to press Second, regarding locking mechanisms, machine
a button on the DRE to print the next page; press- types 1, 3, and 4 publicly exposed part of the printer
ing the button on an audio-assisted voting panel pro- cable, which could be tampered with to disrupt func-
duced no results. Assuming audio-assisted voters are tions during the election. Machine type 2 had no
visually impaired, they’re unlikely to see the message locking mechanism for the printer cover.

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 35

 E-voting

Table 2. Problems found in four tested machine types.

MACHINE MACHINE MACHINE MACHINE

CATEGORY CRITERIA PROBLEMS/CRITERIA EXCEPTIONS TYPE 1 TYPE 2 TYPE 3 TYPE 4
1.1.1 II.B.1, III.B.1, Voter privacy ✓ ✗ ✗ ✗
IV.C.2 (casting/reviewing/verifying)
1.1.2 II.B.4, II.11.b Voter privacy (linkage) ✓ ✓ ✓ ✗
1.1.3 IV.C.2 Voter privacy (audio-assisted voting) ✓ ✗ ✓ ✗
1.1.4 IV.C.4 Voter privacy (alternative language) Not tested Not tested Not tested Not tested
1.2.1 II.B.5, II.B.2.a Record privacy (creation and storage) ✓ ✓ ✓ ✗
1.2.2 IV.B.1, IV.B.1.a Linkage between paper and electronic records ✓ ✗ ✗ ✗
1.2.3 VIII.F Record privacy (auditing) Not tested Not tested Not tested Not tested
2.1.1 Federal criteria DRE’s power reliance Battery Battery Battery Battery
reserves, up reserves, up reserves, reserves, up
to 16 hours to 16 hours up to two to 12 hours
hours
2.1.2 V.F, V.G, V.H, DRE’s error detection and notification ✗ ✗ ✗ ✗
V.I
2.1.3 V.D, V.D.1 Cryptography Not tested Not tested Not tested Not tested
2.1.4 V.H, VIII.D DRE’s internal audit log ✓ ✗ ✗ ✓
2.2.1 II.B.7 VVPRS’s power reliance ✓ ✓ ✓ ✓
2.2.2 III.A.1.a, III.D.1 VVPRS security upon supply replacements ✗ ✓ ✗ ✗
2.2.3 III.A.3, V.C, V.E VVPRS’s locking mechanisms ✗ ✗ ✗ ✗
2.2.4 III.B.3 Paper record display unit ✓ ✓ ✓ ✓
2.2.5 IV.C.5.a.(4) , VVPRS’s error detection and notification ✗ ✓ ✗ ✗
V.G
2.2.6 V.A, V.B VVPRS’s connectivity ✓ ✓ ✓ ✓
2.3.1 IV.A.2.b, Digital signature ✗ ✗ ✗ ✗
IV.B.2, IV.B.3.d
2.3.2 IV.A.6, VIII.E.1 Error correction code ✓ ✓ ✓ ✓
2.3.3 V.K.1 Malfunction recovery for records ✓ ✓ ✓ ✓
3.1.1 II.B.11.a, Verification (prior to casting) ✓ ✗ ✗ ✗
IV.C.5.a.(2)
3.1.2 II.B.11.a, III.B.2 Verification (of entire paper record) ✓ ✓ ✓ ✗
3.1.3 IV.C.5.a, Verification (chance to reject/ ✓ ✓ ✓ ✗
IV.C.5.a.(3) modify upon rejection)
3.2 IV.C.5.a.(1) Recasting ✓ ✓ ✓ ✗
4.1.1 II.11.c, IV.A.1, Paper record components ✗ ✗ ✓ ✓
IV.A.2, IV.A.4,
IV.A.5
4.1.2 III.A.4, III.C.1, Readability of paper records ✓ ✓ ✓ ✗
III.C.3
4.1.3 IV.A.2.c Barcode components in a paper record ✓ ✓ ✓ ✓
4.1.4 IV.C.5, Distinction among accepted and ✗ ✓ ✓ ✗
IV.C.5.a.(5) unaccepted paper records
4.2.1 II.B.10 Components of electrical record ✓ ✓ ✓ ✓
5.1.1 IV.A.2.a DRE’s barcode format ✓ ✓ ✓ ✓
5.1.2 IV.B.3.a Nonproprietary format of the DRE’s electronic ✓ ✓ ✓ ✓
ballot image records
5.2.1 II.B.6 VVPRS’s accessibility Not tested Not tested Not tested Not tested
5.2.2 III.A.1 VVPRS’s sufficient quantities of supplies ✗ ✗ ✗ ✗

5.2.3 III.A.2 VVPRS’s low supply indication ✗ ✓ ✓ ✗

5.2.4 IV.B VVPRS’s limited functionality ✓ ✓ ✓ ✓
6.1.1 IV.B.3.c Documentation (structure of ✓ ✓ ✓ ✓
ballot image records)

36 IEEE SECURITY & PRIVACY ■ MAY/JUNE 2008

 E-voting

Table 2. Problems found in four tested machine types.

MACHINE MACHINE MACHINE MACHINE

CATEGORY CRITERIA PROBLEMS/CRITERIA EXCEPTIONS TYPE 1 TYPE 2 TYPE 3 TYPE 4
6.1.2 IV.B.3.e Documentation (exporting and reconciling ✓ ✓ ✓ ✗
ballot image records)

6.1.3 VIII.D Documentation (generating a ballot image Not tested Not tested Not tested Not tested
record log)
6.2.1 IV.C.1.a Instructions for election worker training ✓ ✓ ✓ ✓
6.2.2 IV.C.1.b, Instructions for voters Not tested Not tested Not tested Not tested
IV.C.1.d
6.2.3 V.J, V.K, V.L, Documentation (VVPRS malfunction recovery) ✓ ✓ ✓ ✓
VIII.H
6.2.4 VI.C.1, VI.C.2, Certification ✗ ✗ ✗ ✓
VI.G
6.3.1 V.D.2 Cryptographic certification Not tested Not tested Not tested Not tested
6.3.2 V.J, V.K, VIII.G Documentation (DRE malfunction recovery) ✓ ✓ ✓ ✓

7.1 VI.B, VI.C, VII.A, Hardware and software examination Tested Tested Tested Tested
VIII.A VVPRS- VVPRS- VVPRS- VVPRS-
related related related related
hardware hardware hardware hardware
only only only only
7.2 VI.E Source code examination Not tested Not tested Not tested Not tested
A check mark (✓) indicates no problems during testing; a cross mark (✗) indicates problems related to the criteria requirements. Criteria in red italics
were not tested.

Third, regarding error detection and notification, set of electronic records using only the electronic re-
when we disconnected the type 1 machine’s printer cords, not their corresponding digital signatures. The
cable, the VVPRS didn’t send a signal to the offi- type 4 machine didn’t generate a digital signature for
cial. The voter could continue voting and cast the individual electronic records or for the entire set of
vote, but the machine failed to print a paper record. electronic records.
With the type 3 machine, a VVPRS mechanical er-
ror or malfunction didn’t prompt any error message Paper-record verification
or warning signal, it simply froze the system. Type 4’s We found three violations of the requirements in this
VVPRS couldn’t detect a paper jam; the voter could category (Figure 1, subcategory 3.1). First, after voters
cast votes, but the printer kept printing over the same on type 2 and 3 machines rejected their first two pa-
area on the paper, making it illegible. Moreover, if per records, the system wouldn’t let them adequately
the machine’s printer cable was disconnected after verify their third paper record (although it printed,
the voter pressed the “cast vote” button, the machine it displayed for only a few seconds before spooling).
recorded the electronic record, but didn’t print a bar- The type 4 machine printed only one paper record
code on the paper record. per voter; voters could review and verify subsequent
ballots on the DRE screen, but not on the paper re-
Vote record security mechanisms cord. Once they cast their ballots, the machine print-
All four machines violated vote record security re- ed the paper record, but it was rapidly advanced to
quirements (Figure 1, subcategory 2.3) in relation the spool.
to digital signatures. The type 1 machine generated Second, the type 4 machine let voters review and
electronic records’ digital signatures based on the modify each vote selection one-by-one an unlimited
vendor’s proprietary scheme, rather than on the re- number of times. It also immediately printed each
quired one-to-one scheme (that is, one digital sig- modification—that is, each selection, deselection, or
nature for each electronic record). The type 2 and change—line-by-line. However, it didn’t print un-
3 machines also failed to generate individual digital dervotes in the line-by-line printing, and thus vot-
signatures for each electronic record. Thus, all three ers couldn’t verify undervotes on the paper record
machines calculated digital signatures for the entire before casting.

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 37

 E-voting

Finally, the type 4 machine printed only one paper Privacy

record per voter. Consequently, voters couldn’t reject Election officials should issue procedural instructions
the paper record and then modify their ballots. This for poll workers so they can strategically place vulner-
problem also violated the state’s criteria for vote re- able voting systems, and thus avoid privacy violations
casting and paper records (Figure 1, subcategory 3.2) when people loiter near the booth. When mechanical
because it didn’t let voters recast their ballot up to two errors or malfunctions occur that require official as-
additional times. sistance, voter and ballot privacy must be maintained.
To achieve this, developers can design voting systems
Paper-record integrity such that voters can hide vote selections displayed on
We found three violations in this category (Figure 1, the DRE and in the paper-record display unit prior
subcategory 4.1). First, type 1 and 2 machines didn’t to seeking assistance. Also, for certain errors, the offi-
print the election name on the paper record. Second, cial could give the voter troubleshooting instructions
the type 4 machine’s paper-record printout used a while standing outside the booth.
smaller font size than mandated by the criteria. Third,
the type 1 machine didn’t print clear acceptance in- Security
formation—that is, “voided” or “accepted”—on the Election officials should be required to thoroughly ex-
paper record. amine and use physical locking and protective mecha-
nisms. For example, states should require each polling
VVPRS component functionality, place to establish a hierarchical access with respect to
design, and format a custody chain. Also, developers should design the
We found one violation in this category (Figure 1, voting system to protect paper records with two or
subcategory 5.2). When the type 1 machine detected more security layers—one to secure the printer cover,
a low paper supply, the DRE screen displayed an error and another to secure the storage unit.
message, and voters could continue voting and cast To replace a paper supply, election workers should
their ballots. However, the machine didn’t print the have a key that opens the printer cover and lets them
paper record. This continued with subsequent voters, restock the paper, but that key shouldn’t open the lock
and the machine sent no audio or visual signal to the protecting the paper-record storage unit. Each polling
election official. To reset the voting system, we had place should require two or more authorized super-
to shut it down. In the type 4 machine, if the paper visors to be present before officials can access paper
amount reached the minimum limit during a voting records. To increase security, such officials might use
session, the DRE voided the ballot and didn’t let the different keys in escrow to open the storage unit’s
voter complete casting the vote. lock. Officials should secure the physical memory or
cartridge that stores electronic records in the same
Volume testing issues way as the paper-record storage unit.
We observed several paper jams in volume tests. Dur- As we discussed earlier, states should mandate
ing a 1,200-vote simulated test on the type 2 ma- that the digital signature be generated both for each
chine, a paper jam occurred and 56 paper records individual electronic record as well as for the entire
didn’t print. The DRE continued to cast votes elec- election’s set of electronic records. Further, the digi-
tronically without the VVPRS printing the paper re- tal signature generation algorithm should adopt the
cords. Once the paper jam was cleared, the printing latest standards, with proper algorithm parameters
resumed. However, because the 56 paper records were as recommended by the National Institute of Stan-
lost, we couldn’t reconcile the paper and electronic re- dards and Technology.
cords, and thus it was impossible to conduct an audit.
(This phenomenon wouldn’t occur during an official Other issues
election, as poll workers must activate the machine There are three other issues that require clear man-
for each voter to cast his or her vote.) In the type 4 dates. First, regarding verification and recasting, vot-
machine, a paper jam during the 14-hour test resulted ers must be able to take affirmative action—that is,
in paper being torn apart, and selections and barcodes press a “cast vote” button—to verify their third and
didn’t print. final paper record, but they shouldn’t be able to reject,
modify, or recast that third paper record.
Suggestions and solutions Second, accepted and rejected paper records must
On the basis of our experience in this testing project, be clearly distinguishable—that is, they must be clear-
we propose some solutions to mitigate the identified ly marked “Voided” or “Accepted”—to prevent any
problems. Election officials can implement some of confusion during a recount or audit.
these solutions using procedural and operational in- Finally, electronic records are typically protected
structions; others require software/firmware changes. and must be extracted using the vendors’ proprietary

38 IEEE SECURITY & PRIVACY ■ MAY/JUNE 2008

 E-voting

software. An independent testing agency must be able Technology, 18 July 2007; www.nj.gov/oag/elections/
to evaluate this proprietary software to ensure the Hearing-Reports-7.07/NJIT-Advantage-report-7.07.pdf.
proper protection of electronic records. The agency 11. Sequoia AVC Edge Voter-verified Paper Record Assess-
should also rigorously evaluate all DRE system source ment, report to NJ Attorney General, NJ Institute of
codes to identify and reduce any code errors (bugs or Technology, 18 July 2007; www.nj.gov/oag/elections/
vulnerable codes) as well as any malicious codes (such Hearing-Reports-7.07/NJIT-Edge-report-7.07.pdf.
as backdoor codes). 12. Election Systems & Software iVotronic w/RTAL Voter-
verified Paper Record Assessment, report to NJ Attorney
General, NJ Institute of Technology, 26 Sept. 2007; www.

I t’s fair to say that most of the machines we tested

met most of the state’s criteria. Still, our testing
revealed several problems that must be addressed to
nj.gov/oag/elections/Hearing-Reports-7.07/ES&S
_Final_Report_Sept%2026_2007.pdf.

instill public confidence in using DRE with VVPRS. Nirwan Ansari is professor in the Department of Electrical and
Although our testing was applied only to voting sys- Computer Engineering at the New Jersey Institute of Technol-
tems subject to the State of New Jersey’s approval, we ogy. His research interests include various aspects of broad-
believe that other states can apply and tailor our analy- band networks and multimedia communications. Ansari has
sis, methodologies, testing scenarios, and solutions for a PhD in electrical engineering from Purdue University. Con-
their different requirements and needs. tact him at ansari@njit.edu.

Acknowledgments Pitipatana Sakarindr is a PhD candidate in the Department

Our team members—Michell Darer, Dennis O’Brian, of Electrical and Computer Engineering at the New Jersey In-
Chunhua Chen, and Caroline Christensen—contributed stitute of Technology. His research interests include network
significantly to this project, and we appreciate their support. security, trust and reputation, secure group communications,
and cryptography. Sakarindr has an MS in computer engi-
References neering from the New Jersey Institute of Technology. Contact
1. J. Epstein, “Electronic Voting,” Computer, vol. 40, no. him at ps6@njit.edu.
8, 2007, pp. 92–95.
2. T. Kohno et al., “Analysis of an Electronic Voting Sys- Ehsan Haghani is a PhD candidate in the Department of Elec-
tem,” Proc. IEEE Symp. Security and Privacy, IEEE CS trical and Computer Engineering at the New Jersey Institute of
Press, 2004, pp. 27–40. Technology. His research interests include various aspects of
3. R.T. Mercuri, Electronic Vote Tabulation Checks and Bal- wireless networks including resource allocation and cross-lay-
ances, dissertation, Dept. of Computer and Information er design. Haghani has an MS in communications engineer-
Systems, Univ. Pennsylvania, Oct. 2000. ing from Chalmers University of Technology, Sweden. Contact
4. New Jersey Division of Elections, State of New Jersey him at ehsan@njit.edu.
Criteria for Voter-Verified Paper Record for Direct Recording
Electronic Voting Machines, April 2007; www.nj.gov/oag/ Chao Zhang is a PhD candidate in the Department of Electri-
elections/voter_issues/Final-VVPRS-Criteria.pdf. cal and Computer Engineering at the New Jersey Institute of
5. US Election Assistance Commission, 2005 Voluntary Technology. His research interests include security in wireless
Voting System Guidelines, vol. I (version 1), 2005; www. networks, with an emphasis on ad hoc and sensor networks.
eac.gov/voting%20systems/docs/vvsgvolumei.pdf. Zhang has an MS in computer science from the New Jersey
6. US Election Assistance Commission, 2005 Voluntary Institute of Technology, and an MS in physics from Fudan Uni-
Voting System Guidelines, vol. II (version 1), 2005; www. versity, China. Contact him at Chao.Zhang@njit.edu.
eac.gov/voting%20systems/docs/vvsgvolume_ii.pdf.
7. U.S. Election Assistance Commission, 2007 Voluntary Aridaman K. Jain is senior university lecturer in the Division of
Voting System Guidelines Recommendations to the Election Mathematical Sciences at the New Jersey Institute of Technol-
Assistance Commission, Aug. 2007; http://vote.nist.gov/ ogy. His research interests include statistical modeling, sam-
VVSG-0807/Final-TGDC-VVSG-08312007.pdf. pling surveys, design of experiments, and data analysis. Jain
8. California Division of Elections, Top-to-Bottom Review has a PhD in statistics and industrial engineering from Purdue
Reports of the Voting System, July 2007; www.sos.ca.gov/ University. Contact him at jain@njit.edu.
elections/voting_systems/ttbr/.
9. Avante Vote-Trakker Voter-verified Paper Record Assess- Yun Q. Shi is professor in the Department of Electrical and
ment, report to NJ Attorney General, NJ Institute of Computer Engineering at the New Jersey Institute of Technol-
Technology, 18 July 2007; www.nj.gov/oag/elections/ ogy. His research interests include watermarking, data hid-
Hearing-Reports-7.07/NJIT-Avante-report-7.07.pdf. ing, steganalysis, forensics, and multimedia security. Shi has
10. Sequoia AVC Advantage Voter-verified Paper Record As- a PhD in electrical engineering from University of Pittsburgh.
sessment, report to NJ Attorney General, NJ Institute of Contact him at Shi@njit.edu.

www.computer.org/security/ ■ IEEE SECURITY & PRIVACY 39