Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How OpenElections uses LLMs

thescoop.org

Post reply on HN
Print view View original

51–60 of 60 posts

Re: How OpenElections uses LLMs

#51

Post by wizzwizz4 »

arlort wrote: post #50
xp84 wrote: post #23

Earlier quoted context omitted.

I could be wildly off-base, but I wonder if some of these systems are airgapped, and the only way the data comes off of the closed system is via printing, to avoid someone inserting a flash drive full of malware in the guise of "copying the CSV file." Obviously there are or should be technical ways to safely extract data in a digital format, but I can see a little value in the provable safety that airgapping gives yo…

You could always just print a QR code as well if that's the issue

Or print in the OCR-B font.

Re: How OpenElections uses LLMs

#52

Post by dwillis »

ghghgfdfgh wrote: post #46
dwillis wrote: post #30

Earlier quoted context omitted.

Totally reasonable view, and one of our volunteers actually got the law in Kansas changed to mandate electronic publishing of statewide precinct results in a structured format! But finding legislative champions for this issue isn't easy.

I’ve tried using LLM’s to do the same exact thing (turning precinct-level election results into a spreadsheet) and in my experience they worked rather poorly. Less accurate than traditional OCR, and considering how many fixes I had to make, altogether slower than manual entry. The resolution of the page made an outsized difference. It’s nice that you got it to work, but I am skeptical of it as a permanent solution. T…

Skepticism is a necessary trait in this type of work, for sure. I will say that the performance has improved substantially in the past year, and there are still PDFs that require a lot of work.

We went with official precinct results for two main reasons: there are differences between election night and final results (some of them non-trivial) and to make the work more manageable. Agree that historical results are a real problem, and as a PA native I know only too well the errors that the state data contains, which is why we go county-by-county there.

Re: How OpenElections uses LLMs

#53

Post by dwillis »

7moritz7 wrote: post #47
Normal_gaussian wrote: post #40

I'm not convinced. I had Gemini convert a bunch of charity forms yesterday, and the deviation was significant and problematic. Rephrasing questions, inventing new questions, changing the emphasis; it might be performing a lot better for numerical data sets, but it's rare to have one without a meaningful textual component.

Use 2.5 Pro on ai studio, not the gemini app

That's what I did.

Re: How OpenElections uses LLMs

#54

Post by nojito »

Normal_gaussian wrote: post #40

I'm not convinced. I had Gemini convert a bunch of charity forms yesterday, and the deviation was significant and problematic. Rephrasing questions, inventing new questions, changing the emphasis; it might be performing a lot better for numerical data sets, but it's rare to have one without a meaningful textual component.

Not sure I believe this.

I just quickly took a scanned document and the transcription looks good.

https://19january2021snapshot.epa.gov/sites/static/files/201...

https://g.co/gemini/share/d315b4047224

It even got the faded partial date stamp.

Re: How OpenElections uses LLMs

#55

Post by xp84 »

arlort wrote: post #50
xp84 wrote: post #23

Earlier quoted context omitted.

I could be wildly off-base, but I wonder if some of these systems are airgapped, and the only way the data comes off of the closed system is via printing, to avoid someone inserting a flash drive full of malware in the guise of "copying the CSV file." Obviously there are or should be technical ways to safely extract data in a digital format, but I can see a little value in the provable safety that airgapping gives yo…

You could always just print a QR code as well if that's the issue

Sure, but it'd take a literal Act of Congress to force all these states to force all their independent vendors to do a thing, so good luck. And each vendor would probably charge about a million dollars to each state to do the work, in government contracting world. So, probably better to just use AI to OCR them.

Re: How OpenElections uses LLMs

#56

Post by mmcwilliams »

missingcolours wrote: post #38
model-15-DAV wrote: post #33

Earlier quoted context omitted.

I think that we should encourage elections to _not_ be standardized. The problems among various polities in the USA have many different issues and should not be forced to conform to a specific way that elections should be done. This is a social problem and we should not cram it into a technical solution. Legibility of elections should be maintained at the local level, trying to make things legible at a national level…

Not being standardized is fine and even a positive (diversity of technology vendors is a security feature and increases confidence in elections). But producing machine readable outputs of some sort , instead of physical paper and PDFs, is clearly a positive as well.

Physical paper outputs can be machine readable if they are made using fonts with that are more-accurately OCR readable.

Re: How OpenElections uses LLMs

#57

Post by cwmoore »

bastawhiz wrote: post #37
cwmoore wrote: post #32

Earlier quoted context omitted.

Is it so impossible to expect compliance from government entities?

To get all the states and counties using the same standard? Very impossible. That's the very crux of the tenth amendment. We don't even have consistent traffic laws from state to state.

No, by members of the constituencies under their overlapping jurisdictions.

Re: How OpenElections uses LLMs

#58

Post by Normal_gaussian »

nojito wrote: post #54
Normal_gaussian wrote: post #40

I'm not convinced. I had Gemini convert a bunch of charity forms yesterday, and the deviation was significant and problematic. Rephrasing questions, inventing new questions, changing the emphasis; it might be performing a lot better for numerical data sets, but it's rare to have one without a meaningful textual component.

Not sure I believe this. I just quickly took a scanned document and the transcription looks good. https://19january2021snapshot.epa.gov/sites/static/files/201... https://g.co/gemini/share/d315b4047224 It even got the faded partial date stamp.

Well bully for you accusing people of lying.

Thats one of the best scanned documents I've seen in years. Most scanning now is via phone.

Re: How OpenElections uses LLMs

#59

Post by Normal_gaussian »

7moritz7 wrote: post #47
Normal_gaussian wrote: post #40

I'm not convinced. I had Gemini convert a bunch of charity forms yesterday, and the deviation was significant and problematic. Rephrasing questions, inventing new questions, changing the emphasis; it might be performing a lot better for numerical data sets, but it's rare to have one without a meaningful textual component.

Use 2.5 Pro on ai studio, not the gemini app

I did. I was scanning about 400 forms.

Re: How OpenElections uses LLMs

#60

Post by Normal_gaussian »

timschmidt wrote: post #41
Normal_gaussian wrote: post #40

I'm not convinced. I had Gemini convert a bunch of charity forms yesterday, and the deviation was significant and problematic. Rephrasing questions, inventing new questions, changing the emphasis; it might be performing a lot better for numerical data sets, but it's rare to have one without a meaningful textual component.

I've seen similar. I wonder if traditional organizational solutions, like those employed by the US Military or IBM, might be applicable. Redundancy is one of their tools for achieving reliability from unreliable parts. Instead of asking a single LLM to perform the task, ask 10 different LLMs to perform the same task 10 different times and count them like votes.

Yeah, what I did to "solve" my issue was to use several models (4), then where there was any disagreement farm out to humans (2). 60% went to humans in the end.

I suspect if I'd done some corrective transformations before LLM scanning the success rate would have been higher, but the cost threshold of the project didn't warrant it.

Post reply on HN