Is there an ocr with this kind of accuracy, but can run in a mobile device ? Looking for an ocr that can detect texts with high accuracy in realtime, so option of using cloud ocr is not viable.
Mistral OCR
311–320 of 450 posts
Re: Mistral OCR
#312I've found that the stunning OCR results so far were because the models were trained on the example file category. Is that the case here? Or can this recognize various documents?
Re: Mistral OCR
#313Re: Mistral OCR
#314I've found that the stunning OCR results so far were because the models were trained on the example file category. Is that the case here? Or can this recognize various documents?
After four years of "This VLM will solve OCR forever!" posts I've firmly put VLMs in the "useless until 100t parameters" category.
Some day when there is enough internal state and training data that they can recognize tables, images and text we will get a gpt3 like moment that will make regular OCR obsolete.
But that day is very far off and everyone who I've talked with and consulted over using VLMs in their pipeline is better served doing something else specific to their use case.
An example I've used before: https://imgur.com/a/w2XKObh
The reply:
---
To extract the text contained within each box while ignoring the text in sub-boxes, we can follow these steps:
1. Identify the outermost box and extract its text. 2. Move to the next outermost box and extract its text, ignoring any text within its sub-boxes. 3. Continue this process for all boxes.
Let's apply this to the image provided:
1. The outermost box contains the text: "This should be second". 2. The next outermost box (ignoring the sub-box within it) contains the text: "First". 3. The next box contains the text: "And also this". 4. The final box contains the text: "The quick brown fox".
So, the extracted text from each box, ignoring sub-boxes, is:
1. "This should be second" 2. "First" 3. "And also this" 4. "The quick brown fox"
---
As you can plainly see it is _wildly_ wrong and gives you no way to try and recover from those errors.
Re: Mistral OCR
#315I ran a partial benchmark against marker - https://github.com/VikParuchuri/marker . Across 375 samples with LLM as a judge, mistral scores 4.32, and marker 4.41 . Marker can inference between 20 and 120 pages per second on an H100. You can see the samples here - https://huggingface.co/datasets/datalab-to/marker_comparison... . The code for the benchmark is here - https://github.com/VikParuchuri/marker/tree/master/ben…
Thank you for your work on Marker. It is the best OCR for PDFs I’ve found. The markdown conversion can get wonky with tables, but it still does better than anything else I’ve tried
Re: Mistral OCR
#316What's the simple explanation for why these VLM OCRs hallucinate but previous version of OCRs don't?
Re: Mistral OCR
#317Alas, I can't run it locally. So it still doesn't solve the problem of OCR for my PDF archive containing my private data...
Re: Mistral OCR
#318We're approaching the point where OCR becomes "solved" — very exciting! Any legacy vendors providing pure OCR are going to get steamrolled by these VLMs. However IMO, there's still a large gap for businesses in going from raw OCR outputs —> document processing deployed in prod for mission-critical use cases. LLMs and VLMs aren't magic, and anyone who goes in expecting 100% automation is in for a surprise. You still n…
One problem I’ve encountered at my small startup in evaluating OCR technologies is precisely convincing stakeholders that the “human-in-the-loop” part is both unavoidable, and ultimately beneficial. PMs want to hear that an OCR solution will be fully automated out-of-the-box. My gut says that anything offering that is snake-oil, and I try to convey that the OCR solution they want is possible, but if you are unwilling…
It really depends on their fault tolerance. I think there's a ton of useful applications where OCR would be 99.9%, 99%, and even 98% reliable. Skillful product manager can keep these limitations in mind and work around them.
Re: Mistral OCR
#319This is incredibly exciting. I've been pondering/experimenting on a hobby project that makes reading papers and textbooks easier and more effective. Unfortunately the OCR and figure extraction technology just wasn't there yet. This is a game changer. Specifically, this allows you to associate figure references with the actual figure, which would allow me to build a UI that solves the annoying problem of looking for a…
>a UI that solves the annoying problem of looking for a referenced figure on another page, which breaks up the flow of reading. A tangent but this exact issue is what I was frustrated for a long time with pdf reader and reading science papers. Then I found sioyek that pops up a small window when you hover over links (references and equations and figures) and it solved it. Granted, the pdf file must be in right format…
Thanks for the link! Good to know someone is working on something similar.
Re: Mistral OCR
#320It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…
I want to rejoice that OCR is now a "solved" problem, but I feel like hallucinations are just as problematic as the kind of stuff I have to put up with tesseract -- both require careful manual proofreading for an acceptable degree of confidence. I guess I'll have to try it and see for myself just how much better these solutions are for my public domain archive.org Latin language reader & textbook projects.
possibly doing both & diffing the output to spot contested bits?