Strange that LlamaParse is mentioned in the pricing table but not the results. We’ve used them to process a lot of pages and it’s been excellent each time.
Ingesting PDFs and why Gemini 2.0 changes everything
81–90 of 456 posts
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#82The numbers in the blog post seem VERY inaccurate. Quick calculation: Input pricing: Image input in 2.0 Flash is $0.0001935. Let's ignore the prompt. Output pricing: Let's assume 500 token per page, which is $0.0003 Cost per page: $0.0004935 That means 2,026 pages per dollar. Not 6,000! Might still be cheaper than many solutions but I don't see where these numbers are coming from. By the way, image input is much more…
Correct, it's with batching Vertex pricing with slightly lower output tokens per page since a lot of pages are somewhat empty in real world docs - I wanted a fair comparison to providers that charge per page.
Regardless of what assumptions you use - it's still an order of magnitude + improvement over anything else.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#83Orthogonal to this post, but this just highlights the need for a more machine readable PDF alternative.
I get the inertia of the whole world being on PDF. And perhaps we can just eat the cost and let LLMs suffer the burden going forwards. But why not use that LLM coding brain power to create a better overall format?
I mean, do we really see printing things out onto paper something we need to worry about for the next 100 years? It reminds me of the TTY interface at the heart of Linux. There was a time it all made sense, but can we just deprecate it all now?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#84I've been working on something similar the past couple months. A few thoughts:
- A lot of natural chunk boundaries span multiple pages, so you need some 'sliding window' mechanism for the best accuracy.
- Passing the entire document hurts throughput too much due to the quadratic complexity of attention. Outputs are also much worse when you use too much context.
- Bounding boxes can be solved by first generating boxes using tradition OCR / layout recognition, then passing that data to the LLM. The LLM can then link it's outputs to the boxes. Unfortunately getting this reliable required a custom sampler so proprietary models like Gemini are out of the question.
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#85How does the Gemini OCR perform against non-English language text?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#86Has anyone in the AEC industry who's reading this worked out a good way to get Bluebeam MEP, electrical layouts into Revit (LOD 200-300).
Have seen MarkupX as a paid option, but it seems some AI in the loop can greatly speed up exception handling, encode family placement to certain elevations based on building code docs....
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#87I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
This is a big aha moment for me. If Gemini can do semantic chunking at the same time as extraction, all for so cheap and with nearly perfect accuracy, and without brittle prompting incantation magic, this is huge.
If I used Gemini 2.0 for extraction and chunking to feed into a RAG that I maintain on my local network, then what sort of locally-hosted LLM would I need to gain meaningful insights from my knowledge base? Would a 13B parameter model be sufficient?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#88Earlier quoted context omitted.
How do today’s LLM’s like Gemini compare with the Document Understanding services google/aws/azure have offered for a few years, particularly when dealing with known forms? I think Google’s is Document AI.
I've found the highest accuracy solution is to OCR with one of the dedicated models then feed that text and the original image into an LLM with a prompt like: "Correct errors in this OCR transcription".
How does it behave if the body of text is offensive or what if it is talking about a recipe to purify UF-6 gas at home? Will it stop doing what it is doing and enter lecturing mode?
I am asking not to be cynical but because of my limited experience with using LLMs for any task that may operate on offensive or unknown input seems to get triggered by all sorts of unpredictable moral judgements and dragged into generating not the output I wanted, at all.
If I am asking this black box to give me a JSON output containing keywords for a certain text, if it happens to be offensive, it refuses to do that.
How does one tackle with that?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#89This is super interesting.
Would this be suitable for ingesting and parsing wildly variable unstructured data into a structured schema?
Re: Ingesting PDFs and why Gemini 2.0 changes everything
#90I think very soon a new model will destroy whatever startups and services are built around document ingestion. As in a model that can take in a pdf page as a image and transcribe it to text with near perfect accuracy.
Extracting plain text isn’t that much of a problem, relatively speaking. It’s interpreting more complex elements like nested lists, tables, side bars, footnotes/endnotes, cross-references, images and diagrams where things get challenging.