Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The drama in trying to convert election PDFs to Spreadsheets

markessien.com

Post reply on HN
Print view View original

131–140 of 148 posts

Re: The drama in trying to convert election PDFs to Spreadsheets

#131

Post by olabyne »

mtrovo wrote: post #51

Is the access to the original photos open? It might be fit for a good Kaggle competition, although maybe a little too late for this current election.

If you want, you have exactly the same issue to solve with Kenya last year.

The pictures of all of the voting sites are available, but the country went to chaos to pick a winner. It is crazy , because on the lower level (in voting offices), the vote process was respected and the numbers are trustworthy, but the higher you go and the more corruption happens, as each aggregation of data removes trust to the system.

Re: The drama in trying to convert election PDFs to Spreadsheets

#132

Post by error503 »

crazygringo wrote: post #76

First of all, what a fantastic and inspiring read. But, I'm left greatly confused -- the article never states whether this changed the result. It says that halfway through counting Obi was in the lead, but nothing about when finished counting. And when I look at the spreadsheet, the last row (#3380) appears to be the totals, which lists: APC LP PDP NNPP 149014 85748 329030 8305 Which shows LP (Obi) in third place, ju…

I collected all the _crosschecked CSVs and got:

  LP       PDP      APC      NNPP
  4731127  4555334  5928825  1019045
Obi seems to make second place here, but far from first.

https://i.imgur.com/UaZbXz6.png

Re: The drama in trying to convert election PDFs to Spreadsheets

#133

Post by logifail »

SkeuomorphicBee wrote: post #94
pjc50 wrote: post #87

Earlier quoted context omitted.

How do you recount electronic-only elections?

By looking at the receipts printed by the ballot machines. Ballot machines print either a final tally at the end of the day, or print every single vote and automatically drop it into a physical ballot, depending on the threat model of the country in question. Either way the you have partial or total recount.

> By looking at the receipts printed by the ballot machines.

Let's the clear, you're not really "recounting" the ballots at that point. If the machine is compromised - and we're discussing a situation in which we know CCTV was removed and people were then shot - you have no real idea if the receipt corresponds to the voter's original intent. Or, indeed, if all the receipts from all the voters make it as far as the recount (?)

> Ballot machines print either a final tally at the end of the day, or print every single vote and automatically drop it into a physical ballot, depending on the threat model of the country in question.

How is reprinting the final automated tally supposed to represent a "recount" of the original automated tally?

> Either way the you have partial or total recount.

You really don't. Bits of paper and Mark I eyeballs all the way.

As Tom Scott puts it, "The key point is not is that paper voting is perfect - it isn't - but attacks against it don't scale well"[0].

[0] Why Electronic Voting Is Still A Bad Idea: https://www.youtube.com/watch?v=LkH2r-sNjQs

Re: The drama in trying to convert election PDFs to Spreadsheets

#134

Post by jxramos »

segfaultbuserr wrote: post #47
SergeAx wrote: post #27

Pdf is a very unfortunate format. It is proprietary, it is paper-oriented, its almost single goal is to keep precise printing layout. But for the last 30 years world didn't come up with anything that could compete.

PDF isn't the actual problem in this particular case. The documents here are photographs taken at different camera angles, embedded in PDFs.

I was going to say, using alt drag to select vertical columns is usually how I extract useable tables out from pdfs with embedded tables.

Re: The drama in trying to convert election PDFs to Spreadsheets

#136

Post by SkeuomorphicBee »

logifail wrote: post #133
SkeuomorphicBee wrote: post #94

Earlier quoted context omitted.

By looking at the receipts printed by the ballot machines. Ballot machines print either a final tally at the end of the day, or print every single vote and automatically drop it into a physical ballot, depending on the threat model of the country in question. Either way the you have partial or total recount.

> By looking at the receipts printed by the ballot machines. Let's the clear, you're not really "recounting" the ballots at that point. If the machine is compromised - and we're discussing a situation in which we know CCTV was removed and people were then shot - you have no real idea if the receipt corresponds to the voter's original intent. Or, indeed, if all the receipts from all the voters make it as far as the re…

> How is reprinting the final automated tally supposed to represent a "recount" of the original automated tally?

If you want to detect tampering in the central totalling, then all you need is the end of day receipt of each ballot. Exactly like in OP's case.

If you want to detect tampering in a ballot, then you manually recount the individual printed paper votes inside that ballot. That is something that you should do to a random sample of ballots, plus ballots with unusual totals.

> As Tom Scott puts it, "The key point is not is that paper voting is perfect - it isn't - but attacks against it don't scale well"[0].

That is simply not true, large scale paper ballot tampering scales very well to the point of turning elections, and is much easier to pull off because it happens in the fringe where no one is looking (while tampering the electronic system would require pulling your heist in the IT room where everyone is looking).

Re: The drama in trying to convert election PDFs to Spreadsheets

#137

Post by toyg »

lostlogin wrote: post #122
churchill wrote: post #29

Oh, and Mark didn't mention that Bola Ahmed Tinubu was indicted for heroin charges in the US in 2003, forfeited $460k & is just too old to run a democracy this size. Atiku Abubakar (second candidate) was a former VP and the president he served under (Obasanjo) still insists the dude remains a monument to corruption. There's been a coordinated campaign at all levels to rig this election massively and we saw voter inti…

> is just too old to run a democracy this size. Bola Ahmed Tinubu was born 29 March 1952. He is 70. Joe Biden was Born November 20, 1942. He is 80. There are plenty of world leaders that are old and I completely agree with you. Why aren’t there upper age limits? The UK House of Lords, US Congress and US Supreme Court have this problem too.

Until recently, old age was supposed to be a Good Thing in leaders, because experience would (or rather should) provide perspective to go beyond hysterical everyday squabbles and actually produce Laws that could survive in the long run. It would also mean such leaders had gone through challenging jobs, and demonstrated themselves worthy of higher office.

Re: The drama in trying to convert election PDFs to Spreadsheets

#138

Post by pxc »

error503 wrote: post #132
crazygringo wrote: post #76

First of all, what a fantastic and inspiring read. But, I'm left greatly confused -- the article never states whether this changed the result. It says that halfway through counting Obi was in the lead, but nothing about when finished counting. And when I look at the spreadsheet, the last row (#3380) appears to be the totals, which lists: APC LP PDP NNPP 149014 85748 329030 8305 Which shows LP (Obi) in third place, ju…

I collected all the _crosschecked CSVs and got: LP PDP APC NNPP 4731127 4555334 5928825 1019045 Obi seems to make second place here, but far from first. https://i.imgur.com/UaZbXz6.png

There are some conditions under which a presidential election in Nigeria can trigger a runoff election, so that may still matter, right?

Re: The drama in trying to convert election PDFs to Spreadsheets

#139

Post by jxramos »

crazygringo wrote: post #76

First of all, what a fantastic and inspiring read. But, I'm left greatly confused -- the article never states whether this changed the result. It says that halfway through counting Obi was in the lead, but nothing about when finished counting. And when I look at the spreadsheet, the last row (#3380) appears to be the totals, which lists: APC LP PDP NNPP 149014 85748 329030 8305 Which shows LP (Obi) in third place, ju…

probably because it was submitted as evidence and not yet accepted by the courts or wherever things wound up next.

Re: The drama in trying to convert election PDFs to Spreadsheets

#140

Post by jxramos »

didgetmaster wrote: post #129
crazygringo wrote: post #108

Earlier quoted context omitted.

Oh thanks for clarifying. Turns out the link to the folder for all the states is here: https://drive.google.com/drive/folders/173oHgms6wYy5WKz_i3Lh... But there doesn't appear to be any file that calculates the nationwide totals. It just seems like such a strange omission but I'm on mobile and can't add up the numbers from across a ton of different files myself.

I downloaded all the .CSV files from that site and quickly loaded them into a table. It just took a couple minutes, but I didn't stop to verify that there were not duplicate rows across the various files. When I added up the totals, I got: APC - 7,225,399 LP - 5,286,181 PDP - 5,285,900 NNPP - 1,529,575 Note: I was using a beta version of a new database tool I created to do this.

should be something quick to whip up in a few minutes in pandas I'd think assuming the column headers are identical and in the same order. It would translate into a bunch of pandas concat call and with the merged table a value_counts for the column where the vote is retained.
Post reply on HN