Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

391–400 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#391

Post by Karrot_Kream »

panta wrote: post #381
kbyatnal wrote: post #375

Earlier quoted context omitted.

We work with fortune 500s in sensitive industries (healthcare, fintech, etc). Our policies are: - data is never shared between customers - data never gets used for training - we also configure data retention policies to auto-purge after a time period

But how to get these guarantees from the upstream vendors? Or do you run the LLMs on premises?

If you're using LLM APIs there are SLAs from the vendors to make sure your inputs are not used as training data and other guarantees. Generally these endpoints cost more to use (the compliance fee essentially) but they solve the problem.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#392

Post by svieira »

ajcp wrote: post #137
llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

Not sure what service you're basing your calculation on but with Gemmini I've processed 10,000,000+ shipping documents (PDF and PNGs) of every concievable layout in one month at under $1000 and an accuracy rate of between 80-82% (humans were at 66%). The longest part of the development timeline was establishing the accuracy rate and the ingestion pipeline, which itself is massively less complex than what your workflo…

> [with] an accuracy rate of between 80-82% (humans were at 66%)

Was this human-verified in some way? If not, how did you establish the facts-on-the-ground about accuracy?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#393

Post by mehulashah »

(Disclosure, CEO of Aryn (https://aryn.ai/) here)

Good post. VLM models are improving and Gemini 2.0 definitely changes the doc prep and ingestion pipeline across the board.

What we're finding as we work with enterprise customers:

1. Attribution is super important, and VLMs are there yet. Combining them with layout analysis makes for a winning combo.

2. VLMs are great at prompt-based extraction, but if you have document automation and you don't know where in tables you'll be searching or need to reproduce faithfully -- then precise table extraction is important.

3. VLMs will continue to get better, but the price points are a result of economies of scale that document parsing vendors don't get. On the flip side, document parsing vendors have deployment models that Gemini can't reach.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#395

Post by chickenWing »

lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

It is cheaper now, but I wonder if it will continue to be cheaper when companies like Google and OpenAI decide they want to make a profit off of AI, instead of pouring billions of dollars of investment funds into it. By the time that happens, many of the specialized service providers will be out of business and Google will be free to jack up the price.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#396

Post by kbaker »

chickenWing wrote: post #395
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

It is cheaper now, but I wonder if it will continue to be cheaper when companies like Google and OpenAI decide they want to make a profit off of AI, instead of pouring billions of dollars of investment funds into it. By the time that happens, many of the specialized service providers will be out of business and Google will be free to jack up the price.

I use Claude through OpenRouter (with Aider), and was pretty amazed to see that it routes the requests during the same session almost round-robin through Amazon Bedrock, sometimes through Google Vertex, sometimes through Anthropic themselves, all of course using the same underlying model.

Literally whoever has the cheapest compute.

With the speed that AI models are improving these days, it seems like the 'moat' of a better model is only a few months before it is commoditized and goes to the cheapest provider.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#397

Post by mrbungie »

victorbjorklund wrote: post #369
sitkack wrote: post #348

Earlier quoted context omitted.

Software is dead, if it isn't a prompt now, it will be a prompt in 6 months. Most of what we think software is today, will just be a UI. But UIs are also dead.

Can you prompt a salesforce replacement for an org with 100 000 employees?

Yesterday I read an /r/singularity post in awe cus of a screenshot of a lead management platform from OAI in a japan convention supposedly meant a direct threat to SalesForce. Like, yeah sure buddy.

I would say most acceleracionist/AI bulls/etc don't really understand the true essential complexity in software development. LLMs are being seen as a software development silver bullets, and we know what happens with silver bullets.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#398

Post by sitkack »

fragmede wrote: post #382
cpursley wrote: post #350

Earlier quoted context omitted.

Software without data moats, vender lock-in, etc sure will. All the low handing fruit saas is going to get totally obliterated by LLM built-software.

If I'm an autobody shop or some other well-served niche, how unhappy with them do I have to be to decide to find a replacement, either a competitor of theirs that used an LLM, or bring it in house and go off and find a developer to LLM-acceleratedly make me a better shopmonkey? And there are the integrations. I don't own a low hanging fruit SaaS company, but it seems very sticky, and since the established company alr…

Which means that the current vc-software-ecosystem is the walking dead. The front end webdev is now going to do things that previously took a 10 person startup.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#399

Post by sitkack »

mrbungie wrote: post #397
victorbjorklund wrote: post #369

Earlier quoted context omitted.

Can you prompt a salesforce replacement for an org with 100 000 employees?

Yesterday I read an /r/singularity post in awe cus of a screenshot of a lead management platform from OAI in a japan convention supposedly meant a direct threat to SalesForce. Like, yeah sure buddy. I would say most acceleracionist/AI bulls/etc don't really understand the true essential complexity in software development. LLMs are being seen as a software development silver bullets, and we know what happens with silv…

Come back your comment in 18 months.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#400

Post by sitkack »

SketchySeaBeast wrote: post #359
sitkack wrote: post #348

Earlier quoted context omitted.

Software is dead, if it isn't a prompt now, it will be a prompt in 6 months. Most of what we think software is today, will just be a UI. But UIs are also dead.

I wonder about these takes. Have you never worked in a complex system in a large org before? OK, sure, we can parse a PDF reliably now, but now we need to act on that data. We need to store it, make sure it ends up with the right people who need to be notified that the data is available for their review. They then need to make decisions upon that data, possible requiring input from multiple stakeholders. All that bac…

We need to think terms of light cones, not dog and pony take downs of whatever system you are currently running. See where thigns are going.

I have worked in large systems, both in code and people, compilers, massive data processing systems, 10k business units.

Post reply on HN