Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

261–270 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#261

Post by silverliver »

We are driving full speed into a xerox 2.0 moment and this time we are doing so knowingly. At least with xerox, the errors were out of place and easy to detect by a human. I wonder how many innocent people will lose their lives or be falsely incarcerated because of this. I wonder if we will adapt our systems and procedures to account for hallucinations and "85%" accuracy.

And no, outlawing use the use of AI or increasing liability with its use will have next to nothing to deter its misuse and everyone knows it. My heart goes out to the remaining 15%.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#262

Post by esjeon »

kbyatnal wrote: post #95

It's clear that OCR & document parsing are going to be swallowed up by these multimodal models. The best representation of a document at the end of the day is an image. I founded a doc processing company [1] and in our experience, a lot of the difficulty w/ deploying document processing into production is when accuracy requirements are high (> 97%). This is because OCR and parsing is only one part of the problem, and…

I think professional services will continue to use OCRs in one way or another, because it's simply too cheap, fast, and accurate. Perhaps, multi-modal models can help address shortcomings of OCRs, like layout detection and guessing unrecognizable characters.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#263

Post by llm_trw »

woah wrote: post #234
llm_trw wrote: post #210

Earlier quoted context omitted.

The bitter lesson is very little of the sort. If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day. Unfortunately N^N grows rather fast and we have to do all sorts of interesting engineering to make ML calculations complete before the heat death of the universe.

> Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their h…

To solve mnist without mathematical tricks like convolutions or attention heads you would nees 2.5e42 weights. Assuming that you're using 16 bit weights that 5e42 bytes. A yotta byte is 10e24.

That is you'd need 5 exa yotta bytes to solve it.

Currently the whole world has around 200 zetabytes of storage.

I short for the next 120 years mnist will need mathematical tricks to be solved.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#264

Post by happyopossum »

kennyloginz wrote: post #223
Andrex wrote: post #188

Earlier quoted context omitted.

They finally made filing free. So, maybe this century?

Check again, Elon and his Doge team killed that.

No they didn’t, that claim is ridiculously easy to debunk but it has been going around because it fits the narrative.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#265

Post by kettleballroll »

pigscantfly wrote: post #259
brookst wrote: post #252

Earlier quoted context omitted.

I don't think it makes sense? Somewhere there has to be a RNG for that to be true. MOE itself doesn't introduce randomness, and the routing to experts is part of the model weights, not (I think) a separate model.

The samples your input is batched with on the provider's backend vary between calls and sparse mixture of experts routing when implemented for efficient utilization induces competition among tokens with either encouraged or enforced balance of expert usage among tokens in the same fixed-size group. I think it's unknown or at least undisclosed exactly why sequence non-determinism at zero temperature occurs in these pr…

I thought the temperature only affects randomness at the end of the network (when turning embeddings back I to words using the softmax). It cannot influence routing, which is inherently influenced by which examples get batched together (ie, it might depend on other users of the system)

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#267

Post by godapi »

nthingtohide wrote: post #236
itissid wrote: post #124

Earlier quoted context omitted.

Wait isn't there atleast a two step process here one is semantic segmentation followed by a method like texttract for text - to avoid hallucinations? One cannot possibly say that "Text extracted by a multimodal model cannot hallucinate"? > accuracy was like 96% of that of the vendor and price was significantly cheaper. I would like to know how this 96% was tested. If you use a human to do random sample based testing,…

Can confirm using gemini, some figure numbers were hallucinated. I had to cross-check each row to make sure data extracted is correct.

use different models to extract the page and cross check against each other. generally reduces issues alot

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#268

Post by nnurmanov »

devmor wrote: post #72

I think this is one of the few functional applications of LLMs that is really undeniably useful. OCR has always been “untrustworthy” (as in you cannot expect it to be 100% correct and know you must account for that) and we have long used ML algorithms for the process.

It is not OCR to blame, when you have garbage in you should not expect anything of high quality, especially with handwriting and tables and different languages. Even human beings fail to understand some documents (see doctor's prescriptions)

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#269

Post by RamblingCTO »

lazypenguin wrote: post #27

I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…

Successful document processing vendors to use LLMs already. I know this at least of klippa. They have (apparently) fine-tuned models, prompts etc. The biggest issue with using LLMs directly is error handling, validation and "parameter drift"/randomness. This is the typical "I'll build it myself but worse" thing

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#270

Post by pvo50555 »

gapeslape wrote: post #146

In my mind, Gemini 2.0 changes everything because of the incredibly long context (2M tokens on some models), while having strong reasoning capabilities. We are working on compliance solution ( https://fx-lex.com ) and RAG just doesn’t cut it for our use case. Legislation cannot be chunked if you want the model to reason well about it. It’s magical to be able to just throw everything into the model. And the best thing…

What does "throw everything into the model" entail in your context?

How much data are you able to feed into the model in a single prompt and on what hardware, if I may ask?

Post reply on HN