Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

401–410 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#401

Post by collingreen »

sitkack wrote: post #400
SketchySeaBeast wrote: post #359

Earlier quoted context omitted.

I wonder about these takes. Have you never worked in a complex system in a large org before? OK, sure, we can parse a PDF reliably now, but now we need to act on that data. We need to store it, make sure it ends up with the right people who need to be notified that the data is available for their review. They then need to make decisions upon that data, possible requiring input from multiple stakeholders. All that bac…

We need to think terms of light cones, not dog and pony take downs of whatever system you are currently running. See where thigns are going. I have worked in large systems, both in code and people, compilers, massive data processing systems, 10k business units.

I don't know what light cones or dog and pony mean here but I'm interested in your take - would you care to expand a bit on how the future can reshape that very complicated set of steps and humans described in the parent?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#402

Post by collingreen »

sitkack wrote: post #399
mrbungie wrote: post #397

Earlier quoted context omitted.

Yesterday I read an /r/singularity post in awe cus of a screenshot of a lead management platform from OAI in a japan convention supposedly meant a direct threat to SalesForce. Like, yeah sure buddy. I would say most acceleracionist/AI bulls/etc don't really understand the true essential complexity in software development. LLMs are being seen as a software development silver bullets, and we know what happens with silv…

Come back your comment in 18 months.

I assume this is a slap intended to imply that ai actually IS a silver bullet answer to the parent's described problem and in just 18 months they will look back and realize how wrong they are.

Is that what you mean and, if so, is there anything in particular you've seen that leads you to see these problems being solved well or on the 18 month timeline? That sounds interesting to look at to me and I'd love to know more.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#403

Post by knowaveragejoe »

Havoc wrote: post #311
rjcrystal wrote: post #229

I work in healthcare domain, We've had great success converting printed lab reports (95%) to Json format using 1.5-Flash model. This post is really exciting for me. will definitely try out 2.0 models. The struggle which almost every ocr usecase faces is with handwritten documents(doctor prescriptions with bad handwriting) With gemini 1.5 flash we've had ~75-80% percent accuracy (based on random sampling by pharmacist…

How on earth is anyone ok with 75% accuracy in prescriptions context?!? Or medical anything That’s literally insane

Nobody said they're okay with it, nor did they describe what they use the data for.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#404

Post by dilDDoS »

M4v3R wrote: post #323
freezed8 wrote: post #180

(disclaimer I am CEO of llamaindex, which includes LlamaParse) Nice article! We're actively benchmarking Gemini 2.0 right now and if the results are as good as implied by this article, heck we'll adapt and improve upon it. Our goal (and in fact the reason our parser works so well) is to always use and stay on top of the latest SOTA models and tech :) - we blend LLM/VLM tech with best-in-class heuristic techniques. So…

The very first (and probably hand-picked & checked) example on your website [0] suffers from the very problem people are talking about here - in "Fiscal 2024" row it contains an error for CEO CAP column. On the image it says "$234.1" but the parsed result says "$234.4". A small error, but error nonetheless. I wonder if we can ever fix these kind of errors with LLM parsing. [0] https://www.llamaindex.ai/llamaparse

Looks like this was fixed, the parsed result says "$234.1" on my end. I wonder if the error was fixed manually or with another round of LLM parsing?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#406

Post by ajcp »

svieira wrote: post #392
ajcp wrote: post #137

Earlier quoted context omitted.

Not sure what service you're basing your calculation on but with Gemmini I've processed 10,000,000+ shipping documents (PDF and PNGs) of every concievable layout in one month at under $1000 and an accuracy rate of between 80-82% (humans were at 66%). The longest part of the development timeline was establishing the accuracy rate and the ingestion pipeline, which itself is massively less complex than what your workflo…

> [with] an accuracy rate of between 80-82% (humans were at 66%) Was this human-verified in some way? If not, how did you establish the facts-on-the-ground about accuracy?

Yup, unfortunately the only way to know how good an AI is at anything is to do the same way you'd do with a human: build a test that you know the answers to already. That's also why the accuracy evaluation was by far the most time intensive part of the development pipeline as we had to manually build a "ground-truth" dataset that we could evaluate the AI again.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#407

Post by Bluestein »

twelve40 wrote: post #163

Isn't it amazing how one company invented a universally spread format that takes structured data from an editor (except images obviously) and converts it into a completely fucked-up unstructured form that then requires expensive voodoo magic to convert back into structured data.

... and call's it "portable", to boot.-

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#408

Post by riku_iki »

martingoodson wrote: post #318
lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

I work in financial data and our customers would not accept 96% accuracy in the data points we supply. Maybe 99.96%. For most use cases in financial services, accurate data is very important.

so, what solution are you using to extract data with 99.96% accuracy?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#409

Post by llm_trw »

jeswin wrote: post #352
llm_trw wrote: post #101

This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get past perfect 100% high-fidelity images. You use an object detection model trained on documents to find the bounding boxes of each document section as _images_; each bounding box comes…

I feel compelled to reply. You've made a bunch of assumptions, and presented your success (likely with a limited set of table formats) as the one true way to parse PDFs. There's no such thing. In real world usage, many tables are badly misaligned. Headers are off. Lines are missing between rows. Some columns and rows are separated by colors. Cells are merged. Some are imported from Excel. There are dotted sub section…

You're making an even less charitable set of assumptions:

1). I'm incompetent enough to ignore publicly available table benchmarks.

2). I'm incompetent enough to never look at poor quality data.

3). I'm incompetent enough to not create a validation dataset for all models that were available.

Needless to say you're wrong on all three.

My day rate is $400 + taxes per hour if you want to be run through each point and why VLMs like Gemini fail spectacularly and unpredictably when left to their own devices.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#410

Post by pkkkzip »

llm_trw wrote: post #409
jeswin wrote: post #352

Earlier quoted context omitted.

I feel compelled to reply. You've made a bunch of assumptions, and presented your success (likely with a limited set of table formats) as the one true way to parse PDFs. There's no such thing. In real world usage, many tables are badly misaligned. Headers are off. Lines are missing between rows. Some columns and rows are separated by colors. Cells are merged. Some are imported from Excel. There are dotted sub section…

You're making an even less charitable set of assumptions: 1). I'm incompetent enough to ignore publicly available table benchmarks. 2). I'm incompetent enough to never look at poor quality data. 3). I'm incompetent enough to not create a validation dataset for all models that were available. Needless to say you're wrong on all three. My day rate is $400 + taxes per hour if you want to be run through each point and wh…

whoa, this is a really aggressive response. No one is calling you incompetent rather criticizing your assumptions.

> My day rate is $400 + taxes per hour if you want to be run through each point

Great, thanks for sharing.

Post reply on HN