InfoSENTRY  logo
Election TechREPORTS
InfoSENTRY® Services   www.infosentry.com   info@infosentry.com
Issue 8
June 2002                          Privacy Policy

  FEATURE 1:  Distribution of election petitions by Internet in Arkansas.
  FEATURE 2: User Acceptance Testing for Vote Tabulation Systems and Voter Registration Systems.
  FEATURE 3: Multnomah County (OR) Elections Division
  FEATURE 4: RFPs for Election Systems Consulting Services
  FEATURE 5: 2002 Northwest Regional Election Conference Set to Discuss Election Technology Issues

Distribution of Elections Petitions by Internet
 In Arkansas 

Remember how Arsenio Hall would pose an interesting question to his audience, reflectively put his finger on his chin, and say "Hmmm..." Well, an interesting item came out of Arkansas last month that made us say, "Hmmm..."

A fellow in Arkansas started in May circulating a petition to place an issue on the November ballot to legalize certain types of casino gambling in The Natural State. The deadline for return of the petitions is July 5, 2002. Arkansas State CapitolLike many other ballot issue committees, the person behind the effort created a web page. For this one, he used a Member's page on AOL. 

On the page the person placed the following language "UPON APPROVAL OF THIS AMENDMENT BY THE VOTERS" YOU WILL RECEIVE A SHARE OF STOCK FOR EACH SIGNATURE YOU GET ON A PETITION. THIS IS A PROPOSITION AND IS ONLY VALID UPON VOTER APPROVAL OF THIS AMENDMENT." More details and disclosures follow this beginning. The stock is apparently in an organization named "Diamond State Casinos, Limited." Immediately after this language and various legal notices, there are instructions for printing a copy of the petition and for canvassing for valid signatures on the petition. 

Simply put, the web site appears to offer a unique type of compensation to persons who want to gather petition signatures and the web site provides a "download and print" blank version of the petition.  

This was a unique effort that raised several questions for Secretary of State Sharon Priest's Elections Department. Susan Inman, Director of the Elections Department, says that after a detailed legal search, they came to the conclusion that there were no laws prohibiting the use of the Internet to distribute blank petition forms in this manner. It appears that the petitioner had created the proper form for the petition and had it approved by the Attorney General's office. The Internet became simply another method for distributing blank petitions to potential canvassers. However, the State Securities Division determined that the stock option as offered is not legal.

So, the Internet has been enlisted directly to distribute petitions in elections. If a similar situation has not already popped up in your state, you might want to get a jump on determining if this kind of petition distribution fits your election laws and procedures. Now we can wait to see if the Internet will be an effective tool in this process in Arkansas.

Hmmm...

(Thanks for the information provided by Susan Inman, who became the Elections Department Director in the Arkansas Secretary of State's office in October, 2001. We ran a story on the statewide voter registration system in Arkansas in our July 2001 issue of the newsletter. Congratulations, Susan! And thanks to Jeanette Heinbockel, who preceded Susan as Elections Department Director and now works for Secretary of State Priest in another capacity. Jeanette was most helpful in providing information for our earlier article on Arkansas.)


Top of page

User Acceptance Testing for Vote Tabulation Systems and Voter Registration Systems

We see a lot of "pasteurized" acceptance testing for election management software and vote tabulation systems. The system vendor installs the software on your computer or voting system. Then, for a few hours computer screen shots and printed reports move "past your eyes." At that point, you have "accepted" the software by paying an invoice for the full system amount. Changes in the software, even ones to meet original requirements, are considered under "warranty" or "enhancements." Warranty corrections might mean that you wait weeks or months for what you need. Enhancements usually mean you pay for the additional work to get what you thought you would get in the first place. 

"Past Your Eyes" testing occurs for many reasons. One is that vendors do not typically encourage users' to test their systems formally and comprehensively. Their contracts often refer to testing in the form of mock elections or a public tabulation of votes on a "test database." These activities typically last for a day. 

Also, elections offices are not loaded with people interested in or trained in acceptance testing. Given a choice between doing almost anything else and carrying out a software test, almost anything else almost always wins. Interest in testing is so low that most folks will not even make it to this paragraph in an article on user acceptance testing.

Another reason for skipping detailed user acceptance testing is that political leaders and senior managers often like to demonstrate their take command, in-charge style by "ordering" their managers to "cut to the chase" or "get the project on the fast track." They have made the procurement decision, often at a trade show or after a few demonstrations, and they want to move very quickly from Point A to Point B. They view a test as an unnecessary roadblock to getting on with the work. (We remind everyone that high-speed chases and cars on fast tracks often end in crashes.) 

Another reason for a lack of user acceptance testing is that it typically occurs only at the end of the implementation project. Once the vendor gets behind in earlier stages (such as designing and programming the system or producing the voting equipment), project components like documentation, training, and testing often get compressed into a day or two--or are dropped completely. 

This situation of avoiding full user acceptance testing has helped the information technology industry achieve a rate of 80% of implementation projects that are significantly late, significantly over budget, or outside the original project scope. It has helped the industry have massively high overhead and maintenance costs because of poorly tested, poorly documented software. 

However, more savvy elections offices are building independent, formal user acceptance testing into their system project lifecycles. They are willing to commit the time and resources for structured, documented testing. They want to reduce the possibility of unwanted headlines about voter registration software that does not produce accurate lists or vote tabulation systems that deliver incorrect ballots to voters or have difficulty "getting the votes out" of the voting panels. 

The headlines during the past few months of the 2002 primaries have not been kind in a number of jurisdictions where these problems occurred. We will take a strong guess that in very few, if any, of these jurisdictions is there a set of notebooks sitting on the shelf or a CD-ROM in a jewel case that contains a set of user acceptance test documentation generated before the most recent use in the election.

 When you need user acceptance testing. 

1. When you are getting ready to sign a contract for a new voter registration or election management information system. The key is for you to include in your contract with your selected vendor that you have the right (and responsibility) to develop and administer the user acceptance test before paying for the new system. You will not see in most contracts presented to you by vendors a provision for you to carry out a real user acceptance test. "Past your eyes" testing is more the norm in those contracts. 

2. When you receive a new release or upgrade of your voter registration or election management information system. When you receive a new release or an upgrade to your production system, it is possible that there will be significant changes to the underlying computer programs or databases. Such changes are often significant enough to warrant testing to be sure that "regression" has not occurred. You might have seen this kind of situation when a data entry operation or a report worked just fine before the upgrade, but failed or worked erratically after the upgrade. You cannot simply test to see if the new features and functions involved in the upgrade work. You have to test to determine that the entire system works properly. 

3. When you move to a new Direct Recording Electronic (DRE) voting system. Pending federal and state legislation is encouraging everyone to move to DRE voting systems. Vendors and the media tout these systems as "computerized voting systems"--and they are. These systems are significantly more dependent on computer programs and software configurations than are previous generations of voting equipment. For those election jurisdictions that decide to build their own election files and ballot templates in these new systems, you will be implementing a new mission critical information technology in your office. It is every bit as important that you carry out user acceptance testing for these systems as it is for businesses to test new payroll, accounting, customer relationship management, and manufacturing systems. These tests typically occur well before the system goes into actual use.

Purchasing a new DRE voter registration system--with its requirements for configuration management, software maintenance, and user training--is a move to a truly computerized voting system. We have seen these systems give incorrect vote tally results simply because their underlying file directories were improperly installed and maintained. The tabulation hardware and software worked flawlessly...but the results were incorrect. 

The increasing computerization of vote tabulation equipment requires complete, documented user acceptance testing to be certain that the system meets your office's requirements and needs--even if the system has been tested against standards set by national standards organizations. 

The VOTING SYSTEM STANDARDS released by the Federal Election Commission on April 30, 2002 are excellent and establish badly needed standards for voting equipment. Volume II of this document "addresses documentation required to be submitted by the vendor prior to testing, the tests conducted by the Independent Test Authorities (ITA’s), and the products generated by the test process."

 However, there is an analogy in another certification field that will illustrate the situation facing election offices as they get ready to use computerized voting systems that have met these standards and passed ITA tests. Being ISO 900X compliant is a goal for many industries. It means that the company has a number of business processes that are standardized and documented according to widely recognized international process standards. It does not mean that the company provides uniformly excellent products and services or that you will be able to use them if you purchase them. For example, Chairman Kenneth Lay proudly announced in Enron's First Annual Environmental Health and Safety Report that the firm had taken steps to achieve ISO 9000 certification for several of its global operations. 

The VOTING SYSTEM STANDARDS "Voting Systems Performance and Test Standards: Overview" in Volume I indicate that the standards themselves either do not address or partially address the following issues:

  1. "Administrative and managerial practices outside the direct control of the vendor." The report continues to gives examples of these practices as "acquiring, securing, operating and maintaining a voting system."
  2. "Integration with the Voter Registration Database." 
  3. "Commercial Off-the-Shelf (COTS) Products:...such as card readers, printers, or personal computers...or software products (such as operating systems, programming language compilers, or database management systems).
  4. "Human Error Rate vs. System Error Rate."

We would add to this list the ability to retrieve election results files from voting systems to post them quickly on cable television stations or Internet sites. 

So, the national standards and tests for voting systems do not address many of the hardware, software, documentation, network connection, database integration, and operational aspects of how the systems will work in your office. The responsibility for testing to assure that the systems work in your office rests on your shoulders.

(We encourage everyone to get familiar with the national voting system standards by reading them at http://www.fec.gov/pages/vssfinal/overview.doc . Of course, you will need to contact your Chief State Election Official for specific requirements in your state.)

4. When you upgrade from an older DRE system to a new version-or to run under a new operating system. Just like upgrading to a new version of your older DRE system, to a new software release, or to a new type of voting panel, you are in the same situation as if you are upgrading your voter registration system. You cannot assume that what worked for you before will work properly. You still need to carry out your own acceptance test to determine that the upgraded system functions properly and that no "regression" has occurred. 

As your systems become more and more computerized, the responsibility rests on your shoulders to be sure that everything works. Blaming the vendor simply does not satisfy officials, the media, and the voters when voter names are not on poll books, when voters are not allowed to vote, or when votes "cannot be gotten out of the computer." 

Comprehensive, independent user testing before system acceptance and before use on Election Day is simply becoming a necessary part of running an election office because of the systems' complexities. The Mecklenburg County (NC) Board of Elections' system administrator, Daniel Binford, gave this perspective on the value of developing and administering an independent user acceptance test: "The customer needs to own the whole process and that includes acceptance testing. The vendor has a vested interest in designing something that will make their product shine. Charlotte, North Carolina, skyline The customer needs to design something that looks for the weak spots and that will allow them to make informed decisions. User acceptance testing is due diligence. We have proceeded through these procurements carefully and deliberately. The acceptance tests are very useful to us in those efforts. We want a process that anybody can look at and say 'Well done. The system works.' " 

Michael Dickerson, Director of the Mecklenburg County Board of Elections, looks out of his office window over the Charlotte skyline and adds, "The acceptance test also proved that we can work the system, not just that the vendor can work the system. And, we were checking to make sure that the next generation of our system is as good as the earlier generation that we bought." 

So, formal user acceptance tests go well beyond "past your eyes" acceptance tests that are really just system demonstrations. 

  • They provide a foundation upon which you can make a justifiable final acceptance and payment to the system vendor. 
  • They provide you a wealth of documentation for later use about the system you might purchase. 
  • They provide you and your staff with training and the knowledge that they can manage and use the system to support the detailed operations of conducting elections. 

InfoSENTRY's third-party testing services

InfoSENTRY designs, administers, and documents user acceptance tests for a broad range of clients. We have carried out these activities for our election clients, independently from vendor demonstrations and tests, to assist in their implementations of web applications, vote tabulation systems, voter registration systems, and document management systems. We have developed a "User Acceptance Testing Lessons Learned" white paper based on our experience developing and administering acceptance tests. You can get this white paper by going to http://www.infosentry.com/uatp_lessons_learned.pdf . You can get a sample of a single test item we developed as one test in a complete test plan for a voter registration system proposed for implementation in one of our client sites by going to http://www.infosentry.com/uatp_sample_test_item.pdf . (These documents require the free Acrobat Reader, which you can download from http://www.adobe.com .)


Top of page

Feature Election Office: 
Multnomah County (OR) Elections Division

There are approximately 365,000 registered voters in Multnomah County Oregon. The odds are that the vast majority of those people registered or changed their registration while Vicki Ervin, the county's Elections Division Director, was working at the Elections Division. 

From her office just on the east side of the Morrison Street Bridge over the Willamette River in Portland, Vicki recalls that her first day at work for Multnomah elections was just that--the General Election day in November, 1972. Vicki worked that one day as a temporary worker. She came back in 1974, again as a temporary employee, and soon as a permanent staff member. In 1984, Vicki became the Elections Division Director. At the end of this month, Vicki is going to retire. 

Vicki mentions the stress of the job and its increasingly complexity. When she started on the job in the mid-70s, "We had a computerized system for printing poll books...and banks and banks of books and file cabinets for all the voter registration records and other documents." Since then, "It has become a much more complex operation to manage elections. It is becoming much more visible and we have to help the media and the voters understand the technical complexity of what we do."

The transition in Oregon from polling place-based elections to Vote-By-Mail balloting has meant a major change in the way the Multnomah County Elections Division--and all of Oregon's election officials--conduct elections. Vicki was fully involved in the transition and even produced a videotape to instruct the media and voters how the new system would work. 

When asked to identify the single biggest technology change that has come along during her tenure in Multnomah County, Vicki responded quickly, Vicki Ervin looking over a digitized signature"Getting rid of manual voter registration records after they come in and having scanned card images and digitized signatures. That move has replaced a tremendous amount of manual labor and gave us the opportunity for operating efficiencies."

Vicki continues, "We have two fewer people in our office than we did a few years ago.  Still, the number of registered voters continues to climb and Multnomah County's turnout in the 2000 General Election was 78%. Also, we now take Passport applications in our office. Document imaging and document management have allowed us to handle this work with increased efficiencies." 

What is one technical capability that Vicki would like to have that is not in Multnomah County's current arsenal? "We would like to have the voter management system more directly integrated with the vote tally system. That would allow us to enter a name once and have it used by all systems, flowing through the entire process. We planned on getting that when we installed the new voter management system, but it didn't work out."

When asked for some ideas that election officials might have in mind as they approach the coming changes in elections management, Vicki comments that "Elections officials should always be looking at what technology is coming down the road and around the corner. Don’t just focus on what is here today.” In an even more specific vein, she suggests that "...elections offices should not put in new vote tally systems, or even voter registration systems, for first use in a primary or a General Election. Use new systems in a smaller election or a more limited capacity at first."

Vicki's replacement will not have to travel far to get to the job. He is John Kauffman, who has been the County Clerk in neighboring Clackamas County since 1986. Vicki and John mugging for the camera in 2002So, John (shown in this recent photo with Vicki) is not new to elections work in Oregon. He started work as a temporary in the Clackamas County printing plant in 1975 and moved to the elections department in 1977. John announced in December 2001 that he would not run for re-election as County Clerk in 2002. In January, when the position opened for Elections Director in Multnomah County, John applied. He looks forward to starting full time in Multnomah County in the next couple of weeks. Congratulations, John! Some retirement!

We first worked with Vicki in 1998 and 1999 when she served on the Advisory Board for a needs assessment and requirements analysis InfoSENTRY Services prepared for a statewide voter registration system in Oregon. Vicki threw herself into that work, as she has done for the past 28 years in Multnomah County. For some reason, it is simply hard to believe that she will leave elections work completely at the end of this month.


Top of page

RFPs for Election Systems Consulting Services

Please send us your Requests for Proposals (RFPs) and consider us as a potential firm to provide information technology consulting services to your election offices. You can see our full range of services at www.infosentry.com . If you distribute the RFPs by electronic mail attachments, please send them to helen_sims@infosentry.com . If you distribute them by mail, please send them to Helen Sims, InfoSENTRY Services, Inc., P.O. Box 28048, Raleigh, NC 27611.

2002 Northwest Regional Election Conference Set to Discuss Election Technology Issues

The Oregon Association of County Clerks and Washington State Association of County Auditors are sponsoring the 2002 Regional Election Conference in Portland at the Jantzen Beach Doubletree Hotel from July 9 - 12. There are a several panels set that deal with election technology issues. In addition to a session on Vote By Mail there are two breakout sessions that we will call to your attention:

TIMELINESS OF ELECTION RESULTS (Greg Kimsey & Kim Wyman) Panelists: Ann Joyce, Associated Press 
Joni Balter, Seattle Times Editorial Staff 
Julie Anne Kempf, King County Elections Superintendent, Washington
Al Davidson, Marion County Clerk, Oregon 
Kurt Fritz, Director - Washington Senate Democratic Campaign Committee 

CONNECTING THE DOTS - STATEWIDE VOTER REGISTRATION SYSTEMS (Sheryl Moss) 
Panelists: Paul Miller, Initiatives Manager - WA Secretary of State's Office
Scott Smith, Oregon Secretary of State's Office 
Doug Chapin, Electionline.org 
Tim Augustine, Maryland State Elections Division

You can get more info at www.2002nwconf.org


Top of page

Please click here to visit our main election systems consulting page. It has a table of contents for previous newsletter issues.

Please visit our main information technology consulting page and our information technology project management page. They contain brief descriptions of some of our previous consulting engagements, including those for election jurisdictions.

Please click here to visit our main system security and disaster recovery page.

Please contact et@infosentry.com if you would like to get a PDF version or a laser printed copy of this newsletter for distribution in your election office.

InfoSENTRY Services, Inc.
2 Hannover Square, Suite 1740 Raleigh, NC 27601
P.O. Box 28048, Raleigh, NC 27611
Phone: 919.838.8570
Glenn Newkirk's e-mail:
glenn_newkirk@infosentry.com

Copyright 2002, InfoSENTRY® Services, Inc. All fights reserved. Reproduction and dissemination without the express written permission of InfoSENTRY® Services, Inc. is strictly prohibited. InfoSENTRY Services, Inc. publishes Election TechReports monthly, focusing on technology trends and issues in election offices.  From time to time, Election TechReports might mention the name of vendors' hardware or software products. However, InfoSENTRY® Services is completely independent from hardware and software vendors. Mentions of vendors' hardware and software products in no way constitutes an endorsement or indication of worthiness for those vendors or products.