Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

201–210 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#201

Post by dkjaudyeqooe »

sconeguy wrote: post #168
twelve40 wrote: post #163

Isn't it amazing how one company invented a universally spread format that takes structured data from an editor (except images obviously) and converts it into a completely fucked-up unstructured form that then requires expensive voodoo magic to convert back into structured data.

What would it have taken to store the plain text in some meta field in the document. Argh, so annoying.

PDF provide that capability, but editors don't produce it, probably because printing is though OS drivers that don't support it, or PDF generators that don't support it. Or they do support it but users don't know to check that option, or turn it off because it makes PDFs too large.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#202

Post by dkjaudyeqooe »

shermantanktop wrote: post #169
twelve40 wrote: post #163

Isn't it amazing how one company invented a universally spread format that takes structured data from an editor (except images obviously) and converts it into a completely fucked-up unstructured form that then requires expensive voodoo magic to convert back into structured data.

is "put this glyph at coordinate (x,y)" really what you'd call "structured"?

He's calling PDFs unstructured: structured editors -> unstructured PDF -> structured data

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#203

Post by groby_b »

sconeguy wrote: post #168
twelve40 wrote: post #163

Isn't it amazing how one company invented a universally spread format that takes structured data from an editor (except images obviously) and converts it into a completely fucked-up unstructured form that then requires expensive voodoo magic to convert back into structured data.

What would it have taken to store the plain text in some meta field in the document. Argh, so annoying.

PDF supports that just fine. It's just that many PDF publishers choose not to use that.

You can lead a horse to water...

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#205

Post by phoh »

scotty79 wrote: post #155
esafak wrote: post #141

Earlier quoted context omitted.

It's not ironic. PDFs are a container , which can hold scanned documents as well as text. Scanned documents need OCR and to be analyzed for their layout. This is not a failing of the PDF format, but a problem inherent to working with print scans. I don't claim PDF is a good format. It is inscrutable to me.

Pdf is a horrible format. Even if it contains plain text it has no concept of something as simple as paragraphs. One can wonder how much wonkiness of llms comes from errors in extracting language from pdfs. Adobe is the most harmful software development company in existence.

amen

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#206

Post by groby_b »

zoogeny wrote: post #106
layer8 wrote: post #93

Earlier quoted context omitted.

PDF does support incorporating information about the logical document structure, aka Tagged PDF. It’s optional, but recommended for accessibility (e.g. PDF/UA). See chapters 14.7–14.8 in [1]. Processing PDF files as rendered images, as suggested elsewhere in this thread, can actually dramatically lose information present in the PDF. Alternatively, XML document formats and the like do exist. Indeed, HTML was supposed…

I'm not suggesting we re-invent RDF or any other kind of semantic web idea. And the fact that semantic data can be stored in a PDF isn't really the problem being solved by tools such as these. In many cases, PDF is used for things like scanned documents where adding that kind of metadata can't really be done manually - in fact the kinds of tools suggested in the post would be useful for adding that metadata to the PD…

I mean, you want to store a kitchen sink of data, too. You don't like the semantic web or semantic metadata, fine - what do you propose? A custom metadata format for each use case? That is semantic information.

If you don't do that, you get a kitchen sink. If you need to store 1930s death certificats, 10k filings, your doctor's signup forms, the ARR graph for your startup, and a genealogy chart all in the same format, kitchen sink it is.

If it were "just a wrapper for image data", what exactly would that wrapper add? Semantic information, or a kitchen sink to manage additional info.

You're asking to store complex data without preserving complexity - I don't think that'll work.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#207

Post by bdd8f1df777b »

lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

What if you prompt Gemini that mistaking LLC for IIC is a common mistake? Will Gemini auto correct it?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#208

Post by tomrod »

bdd8f1df777b wrote: post #207
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

What if you prompt Gemini that mistaking LLC for IIC is a common mistake? Will Gemini auto correct it?

With lower temperature, it seems to work okay for me.

A _killer_ awesome thing it does too is allow code specification in the config instead of through repeated attempts at prompts.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#209

Post by nyarlathotep_ »

themanmaran wrote: post #129
itissid wrote: post #124

Earlier quoted context omitted.

Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…

One thing people always forget about traditional OCR providers (azure, tesseract, aws textract, etc.) is that they're ~85% accurate. They are all probabilistic. You literally get back characters + confidence intervals. So when textract gives you back incorrect characters, is that a hallucination?

> is that they're ~85% accurate.

Speaking from experience, you need to double check "I" and "l" and "1" "0" and "O" all the time, accuracy seems to depend on the font and some other factors.

have a util script I use locally to copy some token values out of screenshots from a VMWare client (long story) and I have to manually adjust 9/times.

How relevant that is or isn't depends on the use case.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#210

Post by llm_trw »

woah wrote: post #190
llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Getting "bitter lesson" vibes from this post

The bitter lesson is very little of the sort.

If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day.

Unfortunately N^N grows rather fast and we have to do all sorts of interesting engineering to make ML calculations complete before the heat death of the universe.

Post reply on HN