91–100 of 456 posts
I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
I work in fintech and we replaced an OCR vendor with Gemini at work for ingesting some PDFs. After trial and error with different models Gemini won because it was so darn easy to use and it worked with minimal effort. I think one shouldn't underestimate that multi-modal, large context window model in terms of ease-of-use. Ironically this vendor is the best known and most successful vendor for OCR'ing this specific ty…
Out of interest, did you parse into any sort of defined schema/structure?
> Our prompt is currently very simple: "OCR this PDF into this format as specified by this json schema" and didn't require some fancy "prompt engineering" to contort out a result.
Orthogonal to this post, but this just highlights the need for a more machine readable PDF alternative. I get the inertia of the whole world being on PDF. And perhaps we can just eat the cost and let LLMs suffer the burden going forwards. But why not use that LLM coding brain power to create a better overall format? I mean, do we really see printing things out onto paper something we need to worry about for the next…
Alternatively, XML document formats and the like do exist. Indeed, HTML was supposed to be a document format. That’s not the problem. The problem is having people and systems actually author documents in that way in an unambiguous fashion, and having a uniform visual presentation for it that would be durable in the long term (decades at least).
PDF as a format persists because it supports virtually every feature under the sun (if authors care to use them), while largely guaranteeing a precisely defined visual presentation, and being one of the most stable formats.
[1] https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandard...
Been toying with the flash model. Not the top model, but think it'll see plenty use due to the details. Wins on things other than top of benchmark logs * Generous free tier * Huge context window * Lite version feels basically instant However * Lite model seems more prone to repeating itself / looping * Very confusing naming e.g. {model}-latest worked for 1.5 but now its {model}-001? The lite has a date appended, the…
> * Huge context window But how well does it actually handle that context window? E.g. a lot of models support 200K context, but the LLM can only really work with ~80K or so of it before it starts to get confused.
Recent Gemini models actually do extraordinarily well.
https://cloud.google.com/blog/products/ai-machine-learning/t...
I founded a doc processing company [1] and in our experience, a lot of the difficulty w/ deploying document processing into production is when accuracy requirements are high (> 97%). This is because OCR and parsing is only one part of the problem, and real world use cases need to bridge the gap between raw outputs and production-ready data.
This requires things like:
- state-of-the-art parsing powered by VLMs and OCR
- multi-step extraction powered by semantic chunking, bounding boxes, and citations
- processing modes for document parsing, classification, extraction, and splitting (e.g. long documents, or multi-document packages)
- tooling that lets nontechnical members quickly iterate, review results, and improve accuracy
- evaluation and benchmarking tools
- fine-tuning pipelines that turn reviewed corrections —> custom models
Very excited to get test and benchmark Gemini 2.0 in our product, very excited about the progress here.
I have a scanner, and some OCR processes I run things through. I am close to 85% from my automatic process.
The pain of going from 85% to 99% though is considerable. (and in my case manual) (well Perl helps)
I went to try this AI on one of the short poem manufscript I have.
I told the prompt I wanted PDF to Markdown, it says sure go ahead give me the pdf. I went upload it. It spent a long time spinning. then a quick messages comes up, something like
"Failed to count tokens"
but it just flashes and goes away.
I guess the PDF is too big? Weird though, its not a lot of pages.
Earlier quoted context omitted.
Modern multimodal encoders for LLMs are fine/not lossy since they do not resize to a small size and can handle arbitrary sizes, although some sizes are obviously better represented in the training set. A 8.5" x 11" paper would be common. I suspect the issue is prompt engineering related. > Please provide me strict bounding boxes that encompasses the following text in the attached image? I'm trying to draw a rectangle…
Just tried this and it did not appear to work for me. Prompt: >Please provide me strict bounding boxes that encompasses the following text in the attached image? I'm trying to draw a rectangle around the text. > - Use the top-left coordinate system >this input document is 1080 x 1236 px. return the bounding boxes as integers
They say there's no magic prompt but I'd start with their default since there is usually some format used to improve performance with posttraining with tasks like this
Earlier quoted context omitted.
Small point but is it doing semantic chunking, or loading the entire pdf into context? I've heard mixed results on semantic chunking.
It loads the entire PDF into context, but then it would be my job to chunk the output for RAG, and just doing arbitrary fixed-size blocks, or breaking on sentences or paragraphs is not ideal. So I can ask Gemini to return chunks of variable size, where each chunk is a one complete idea or concept, without arbitrarily chopping a logical semantic segment into multiple chunks.
How about 2.0.2?
How about Llama 13.4.0.1?
This is tiring. It's always the end of the world when they release a new version of some LLM.