Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

251–260 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#251

Post by eitally »

wussboy wrote: post #165
panarky wrote: post #33

Earlier quoted context omitted.

This is a big aha moment for me. If Gemini can do semantic chunking at the same time as extraction, all for so cheap and with nearly perfect accuracy, and without brittle prompting incantation magic, this is huge.

Could it do exactly the same with a web page? Would this replace something like beautiful soup?

I don't know exactly how or what it's doing behind the scenes, but I've been massively impressed with the results Gemini's Deep Research mode has generated, including both traditional LLM freeform & bulleted output, but also tabular data that had to come from somewhere. I haven't tried cross-checking for accuracy but the reports do come with linked sources; my current estimation is that they're at least as good as a typical analyst at a consulting firm would create as a first draft.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#252

Post by brookst »

petesergeant wrote: post #247
brookst wrote: post #246

Earlier quoted context omitted.

If temperature is zero, and weights are weights, where is the non-deterministic behavior coming from?

The parent is suggesting that temperature only applies at the generation step, but the choice of backend “expert model” that a request is given to (and then performs the generation) is non-deterministic. Rather than being a single set of weights, there are a few different sets of weights that constitute the “expert” in MoE. I have no idea if that’s true, but that’s the assertion

I don't think it makes sense? Somewhere there has to be a RNG for that to be true. MOE itself doesn't introduce randomness, and the routing to experts is part of the model weights, not (I think) a separate model.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#253

Post by eitally »

llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Fwiw, I'm not convinced Gemini isn't using an document-based objection detection model for this, at least some parts of this or for some doc categories (especially common things like IDs, bills, tax forms, invoices & POs, shipping documents, etc that they've previously created document extractors for (as part of their DocAI cloud service).

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#254

Post by visarga »

layer8 wrote: post #90
nickandbro wrote: post #49

I think very soon a new model will destroy whatever startups and services are built around document ingestion. As in a model that can take in a pdf page as a image and transcribe it to text with near perfect accuracy.

Extracting plain text isn’t that much of a problem, relatively speaking. It’s interpreting more complex elements like nested lists, tables, side bars, footnotes/endnotes, cross-references, images and diagrams where things get challenging.

OCR is not 100% either. Reading order is also fragile, it might OCR the word but mess up the line structure.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#255

Post by simonw »

eitally wrote: post #253
llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Fwiw, I'm not convinced Gemini isn't using an document-based objection detection model for this, at least some parts of this or for some doc categories (especially common things like IDs, bills, tax forms, invoices & POs, shipping documents, etc that they've previously created document extractors for (as part of their DocAI cloud service).

I don't see why they would do that. The whole point of training a model like Gemini is that you train the model - if they want it to work great against those different categories of document the likely way to do it is to add a whole bunch of those documents to Gemini's regular training set.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#256

Post by eternauta3k »

j_timberlake wrote: post #184
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

This sounds extremely like my old tax accounting job. OCR existed and "worked" but it was faster to just enter the numbers manually than fix all the errors. Also, the real solution to the problem should have been for the IRS to just pre-fill tax returns with all the accounting data that they obviously already have. But that would require the government to care.

Germany (not exactly the cradle of digitalization) already auto-fills salary tax fields with data from the employer.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#257

Post by visarga »

matthest wrote: post #51

This is completely tangential, but does anyone know if AI is creating any new jobs? Thinking of the OCR vendors who get replaced. Where might they go? One thing I can think of is that AI could help the space industry take off. But wondering if there are any concrete examples of new jobs being created.

> Thinking of the OCR vendors who get replaced. Where might they go?

We are solving more complicated document types, in more languages, longer in size. The scope of work expanded a lot.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#258

Post by nnurmanov »

depr wrote: post #78
nickandbro wrote: post #49

I think very soon a new model will destroy whatever startups and services are built around document ingestion. As in a model that can take in a pdf page as a image and transcribe it to text with near perfect accuracy.

I think the Azure Document Intelligence, Google Document AI and Amazon Textract are among the best if not the best services though and they offer these models.

I have not tested Azure Document Intelligence, Google Document AI, but AWS Textract, LLamaparse, Unstructured and Omni made to my shortlist. I have not tested Docling, as I could not install it on my Windows laptop.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#259

Post by pigscantfly »

brookst wrote: post #252
petesergeant wrote: post #247

Earlier quoted context omitted.

The parent is suggesting that temperature only applies at the generation step, but the choice of backend “expert model” that a request is given to (and then performs the generation) is non-deterministic. Rather than being a single set of weights, there are a few different sets of weights that constitute the “expert” in MoE. I have no idea if that’s true, but that’s the assertion

I don't think it makes sense? Somewhere there has to be a RNG for that to be true. MOE itself doesn't introduce randomness, and the routing to experts is part of the model weights, not (I think) a separate model.

The samples your input is batched with on the provider's backend vary between calls and sparse mixture of experts routing when implemented for efficient utilization induces competition among tokens with either encouraged or enforced balance of expert usage among tokens in the same fixed-size group. I think it's unknown or at least undisclosed exactly why sequence non-determinism at zero temperature occurs in these proprietary implementations, but I think this is a good theory.

[1] https://arxiv.org/abs/2308.00951 pg. 4 [2] https://152334h.github.io/blog/non-determinism-in-gpt-4/

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#260

Post by llm_trw »

siva7 wrote: post #230
llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

You‘re describing yesterdays world. With the advancement of AI, there is no need for any of these many steps and stages of OCR anymore. There is no need for XML in your pipeline because Markdown is now equally suited for machine consumption by AI models.

The results we got 18 months ago are still better than the current gemini benchmarks at a fraction the cost.

As for markdown, great. Now how do you encode the meta data about the confidence of the model that the text says what it thinks it says? Becuase xml has this lovely thing called attributes that let's you keep a provenance record without a second database that's also readable by the llm.

Post reply on HN