Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

211–220 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#211

Post by minimalengineer »

Two years ago, I worked for a company that had its own proprietary AI system for processing PDFs. While the system handled document ingestion, its real value was in extracting and analyzing data to provide various insights. However, one key requirement was rendering documents in HTML with as close to a 1:1 likeness as possible.

At the time, I evaluated multiple SDKs for both OCR and non-OCR PDF conversions, but none matched the accuracy of Adobe Acrobat’s built-in solution. In fact, at one point (don’t laugh), the company resorted to running Adobe Acrobat on a Windows machine with automation tools to handle the conversion. Using Adobe’s cloud service for conversion was not an option due to the proprietary nature of the PDFs. Additionally, its results were inconsistent and often worse compared to the desktop version of Adobe Acrobat!

Given that experience, I see this primarily as an HTML/text conversion challenge. If Gemini 2.0 truly improves upon existing solutions, it would be interesting to see a direct comparison against popular proprietary tools in terms of accuracy.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#212

Post by stigz »

ThinkBeat wrote: post #96

Hmm I have been doing a but if this manually lately for a personal project. I am working on some old books that are far past any copyright, but they are not available anywhere on the net. (Being in Norwegian m makes a book a lot more obscure) so I have been working on creating ebooks out of them. I have a scanner, and some OCR processes I run things through. I am close to 85% from my automatic process. The pain of go…

I experienced something similar. My use case is I need to summarize bank statements (sums, averages, etc.). Gemini wouldn't do it, it said too many pages. When I asked the max number of supported pages, it says max is 14 pages. Attempted on both 2.0 flash and 2.0 pro in VertexAI console.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#213

Post by pkkkzip »

llm_trw wrote: post #210
woah wrote: post #190

Earlier quoted context omitted.

Getting "bitter lesson" vibes from this post

The bitter lesson is very little of the sort. If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day. Unfortunately N^N grows rather fast and we have to do all sorts of interesting engineering to make ML calculations complete before the heat death of the universe.

I think you are being pedantic here and business decisions aren't made based on purely cost but brittleness, maintenance, time to market.

You are assuming you can match Gemini's performance, Google's engineering resources and costs being constant in to the future.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#214

Post by mnky9800n »

lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

What are the pdfs containing?

I’ve been wanting to build a system that ingests pdf reports that reference other types of data like images, csv, etc. that can also be ingested to ultimately build an analytics database from the stack of unsorted data AB’s meta data but I have not found any time to do anything like that yet. What kind of tooling do you use to build your data pipelines?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#215

Post by lyjackal »

If the end goal is just rag or search over the pdfs, seems like ColPali based embedding search would be a good alternative here. Don’t process the PDFs, instead just search their image embedding directly. From what I understand, you also get a sort of attention as to what part of the image is being activated by the search.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#216

Post by disqard »

pbronez wrote: post #153
oedemis wrote: post #103

there is also https://ds4sd.github.io/docling/ from ibm research which is mit license and track bounding boxes as rich json format

Docling has worked well for me. It handles scenarios that crashed ChatGPT Pro. Only problem is it's super annoying to install. When I have a minute I might package it for homebrew.

Did you compare it to tesseract?

If it's superior (esp. for scans with text flowing around image boxes), and if you do end up packaging it up for brew, know that there's at least one developer who will benefit from your work (for a side-project, but that goes without saying).

Thanks in advance!

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#217

Post by galvin »

Somewhat tangential, but the EU has a directive mandating electronic invoicing for public procurement.

One of the standards that has come out of that is EN 16931, also known as ZUGFeRD and Factur-X, which basically involves embedding an XML file with the invoice details inside a PDF/A. It allows the PDF to be used like a regular PDF but it also allows the government procurement platforms to reliably parse the contents without any kind of intelligence.

It seems like a nice solution that would solve a lot of issues with ingesting PDFs for accounting if everyone somehow managed to agree a standard. Maybe if EN 16931 becomes more broadly available it might start getting used in the private sector too.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#218

Post by llm_trw »

pkkkzip wrote: post #213
llm_trw wrote: post #210

Earlier quoted context omitted.

The bitter lesson is very little of the sort. If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day. Unfortunately N^N grows rather fast and we have to do all sorts of interesting engineering to make ML calculations complete before the heat death of the universe.

I think you are being pedantic here and business decisions aren't made based on purely cost but brittleness, maintenance, time to market. You are assuming you can match Gemini's performance, Google's engineering resources and costs being constant in to the future.

>You are assuming you can match Gemini's performance

I'm not assuming. We already did, 18 months ago with better performance than the current generation of Gemini for our use case.

You're falling into the usual trap of thinking that because big tech spends big money it gets big results. It doesn't. To quote a friend who was a manager at google "If only I could get my team of 100 to be as productive as my first team of three.".

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#219

Post by hattmall »

panarky wrote: post #33
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

This is a big aha moment for me. If Gemini can do semantic chunking at the same time as extraction, all for so cheap and with nearly perfect accuracy, and without brittle prompting incantation magic, this is huge.

It's cheap now because Google is subsidizing it, no?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#220

Post by vrosas »

hattmall wrote: post #219
panarky wrote: post #33

Earlier quoted context omitted.

This is a big aha moment for me. If Gemini can do semantic chunking at the same time as extraction, all for so cheap and with nearly perfect accuracy, and without brittle prompting incantation magic, this is huge.

It's cheap now because Google is subsidizing it, no?

Spoiler: every model is deeply, deeply subsidized. At least Google's is subsidized by a real business with revenue, not VC's staring at the clock.
Post reply on HN