Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ingesting PDFs and why Gemini 2.0 changes everything

sergey.fyi

Post reply on HN
Print view View original

291–300 of 456 posts

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#291

Post by myaccountonhn »

rjcrystal wrote: post #229

I work in healthcare domain, We've had great success converting printed lab reports (95%) to Json format using 1.5-Flash model. This post is really exciting for me. will definitely try out 2.0 models. The struggle which almost every ocr usecase faces is with handwritten documents(doctor prescriptions with bad handwriting) With gemini 1.5 flash we've had ~75-80% percent accuracy (based on random sampling by pharmacist…

> We've had great success converting printed lab reports (95%) to Json format using 1.5-Flash model

Sounds terrifying. How can you be sure that there were no conversion mistakes?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#292

Post by gapeslape »

pvo50555 wrote: post #270
gapeslape wrote: post #146

In my mind, Gemini 2.0 changes everything because of the incredibly long context (2M tokens on some models), while having strong reasoning capabilities. We are working on compliance solution ( https://fx-lex.com ) and RAG just doesn’t cut it for our use case. Legislation cannot be chunked if you want the model to reason well about it. It’s magical to be able to just throw everything into the model. And the best thing…

What does "throw everything into the model" entail in your context? How much data are you able to feed into the model in a single prompt and on what hardware, if I may ask?

Gemini models run in the cloud, so there is no issue with hardware.

The EU regulations typically include delegated acts, technical standards, implementation standards and guidelines. With Gemini 2.0 we are able to just throw all of this into the model and have it figure out.

This approach gives way better results than anything we are able to achieve with RAG.

My personal bet is that this is how the future will look like. RAG will remain relevant, but only for extremely large document corpuses.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#293

Post by flask_manager »

llm_trw wrote: post #263
woah wrote: post #234

Earlier quoted context omitted.

> Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their h…

To solve mnist without mathematical tricks like convolutions or attention heads you would nees 2.5e42 weights. Assuming that you're using 16 bit weights that 5e42 bytes. A yotta byte is 10e24. That is you'd need 5 exa yotta bytes to solve it. Currently the whole world has around 200 zetabytes of storage. I short for the next 120 years mnist will need mathematical tricks to be solved.

The distinction that i think is important to make when talking about "the bitter lesson" is that improving the compute and training infrastructure and tricks in the abstract wins over intelligent model and system design.

Its more about the information about the specific problem you are solving having less impact than techniques that target the compute. So in this case, breaking down how to parse a PDF in stages for your domain is involving specific expert knowledge of the domain, but training with attention is about efficient use of compute in general; with no domain expertise.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#294

Post by gapeslape »

manmal wrote: post #183
gapeslape wrote: post #146

In my mind, Gemini 2.0 changes everything because of the incredibly long context (2M tokens on some models), while having strong reasoning capabilities. We are working on compliance solution ( https://fx-lex.com ) and RAG just doesn’t cut it for our use case. Legislation cannot be chunked if you want the model to reason well about it. It’s magical to be able to just throw everything into the model. And the best thing…

Maybe a dumb question, have you tried fine tuning on the corpus, and then adding a reasoning process (like all those R1 distillations)?

We haven't tried that, we might do that in the future.

My intuition - not based on any research - is that recall should be a lot better from in context data vs. weights in the model. For our use case, precise recall is paramount.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#295

Post by zelphirkalt »

wodenokoto wrote: post #287
brookst wrote: post #246

Earlier quoted context omitted.

If temperature is zero, and weights are weights, where is the non-deterministic behavior coming from?

Temperature changes the distribution that is sampled, not if a distribution is sampled. Temperature changes the softmax equation[1], not weather or not you are sampling from the softmax result or choosing the highest probability. IBM's documentation corroborates this, saying you need to set do_sample to True in order for the temperature to have any effect, e.g., T changes how we sample, not if we sample [2]. A simila…

I have dealt with traditional ML models in the past and things like tensorflow non-reproducibility. Managed to make them behave reproducibly. This is a very basic requirement. If we cannot even have that or people who deal with Gemini or similar models do not even know why they don't deliver reproducible results ... This seems very bad. It becomes outright unusable for anyone wanting to do research with reliable result. We already have a reproducibility crisis, because researchers often do not have the required knowledge to properly handle their tooling and would need a knowledgeable engineer to set it up. Only that most engineers don't know either and don't show enough attention to the detail to make reproducible software.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#296

Post by nabla9 »

anirudhb99 wrote: post #65

thanks a ton for all the amazing feedback on this thread! if (a) you have document understanding use cases that you'd like to use gemini for (the more aspirational the better) and/or (b) there are loss cases for which gemini doesn't work well today, please feel free to email anirudhbaddepu@google.com and we'd love to help get your use case working & improve quality for our next series of model updates!

What if you need scan pages from thick paper books or binded documents without specialized book scanner?

I have two user cases in mind:

1. Photographs of open book.

2. Having video feed of open book where someone flips pages manually.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#297

Post by Dylan16807 »

TeMPOraL wrote: post #275
brookst wrote: post #246

Earlier quoted context omitted.

If temperature is zero, and weights are weights, where is the non-deterministic behavior coming from?

Here probably routing would be dominating, but in general, unless I missed all the vendors ditching GPUs and switching to ASICs optimized for fixed precision math, floating points are still non-commutative therefore results are non-deterministic wrt. randomness introduced by parallelising the calculations.

Why would the same software on the same GPU architecture use different commutations from run to run?

Also if you're even considering fixed point math, you can use integer accumulators to add up your parallel chunks.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#298

Post by zelphirkalt »

TeMPOraL wrote: post #275
brookst wrote: post #246

Earlier quoted context omitted.

If temperature is zero, and weights are weights, where is the non-deterministic behavior coming from?

Here probably routing would be dominating, but in general, unless I missed all the vendors ditching GPUs and switching to ASICs optimized for fixed precision math, floating points are still non-commutative therefore results are non-deterministic wrt. randomness introduced by parallelising the calculations.

Of course which part of the calculations happens where should also be specifiable and be able to be made deterministicor should not have an effect on the result. A map reduce process' reduce step, merging results from various places also should be able to be made to give reproducible results, regardless of which results arrive first or from where.

Is our tooling too bad for this?

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#299

Post by csomar »

silverliver wrote: post #261

We are driving full speed into a xerox 2.0 moment and this time we are doing so knowingly. At least with xerox, the errors were out of place and easy to detect by a human. I wonder how many innocent people will lose their lives or be falsely incarcerated because of this. I wonder if we will adapt our systems and procedures to account for hallucinations and "85%" accuracy. And no, outlawing use the use of AI or increa…

Humans accept a degree of error for convenience. (driving is one of them). But no, 15% is not the acceptable rate. More like 0.15% to 0.015% depending on the country.

Re: Ingesting PDFs and why Gemini 2.0 changes everything

#300

Post by rafaelmn »

mejutoco wrote: post #288
makeitdouble wrote: post #172

Earlier quoted context omitted.

> After trial and error with different models As a mere occasional customer I've been scanning 4 to 5 pages of the same document layout every week in gemini for half a year, and every single week the results were slightly different. To note the docs are bilingual so it could affect the results, but what stroke me is the lack of consistency, and even with the same model, running it two or three times in a row gives di…

> and every single week the results were slightly different. This is one of the reasons why open source offline models will always be part of the solution, if not the whole solution.

Inconsistency comes from scaling - if you are optimizing your infra to be cos effective you will arrive at same tradeoffs. Not saying it's not nice to be able to make some of those decisions on your own - but if you're picking LLMs for simplicity - we are years away from running your own being in the same league for most people.
Post reply on HN