Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral OCR

mistral.ai

Post reply on HN
Print view View original

291–300 of 450 posts

Re: Mistral OCR

#291

Post by devmor »

notepad0x90 wrote: post #212
devmor wrote: post #73

Earlier quoted context omitted.

While those are concerns, my point was that having everything on the internet navigated to, digested and explained to me sounds unpleasant and overall a drain on my ability to think and reason for myself. It is specifically how you describe using the tech that provokes a feeling of revulsion to me.

Then I think you misunderstand. The ML system would know when you want things digested to you or not. Right now companies are assuming this and forcing LLM interaction. But when properly done, the system would know based on your behavior or explicit prompts what you want and provide the service. If you're staring at a paragraph intently and confused, it might start highlighting common phrases or parts of the text/pic…

No, I fully understand.

I am saying that this type of system, that deprives the user of problem solving, is itself a problem. A detriment to the very essence of human intelligence.

Re: Mistral OCR

#292

Post by themanmaran »

themanmaran wrote: post #96
cxie wrote: post #40

The new Mistral OCR release looks impressive - 94.89% overall accuracy and significantly better multilingual support than competitors. As someone who's built document processing systems at scale, I'm curious about the real-world implications. Has anyone tried this on specialized domains like medical or legal documents? The benchmarks are promising, but OCR has always faced challenges with domain-specific terminology…

Excited to test this our on our side as well. We recently built an OCR benchmarking framework specifically for VLMs[1][2], so we'll do a test run today. From our last benchmark run, some of these numbers from Mistral seem a little bit optimistic. Side by side of a few models: model | omni | mistral | gemini | 86% | 89% | azure | 85% | 89% | gpt-4o | 75% | 89% | google | 68% | 83% | Currently adding the Mistral API an…

Update: Just ran our benchmark on the Mistral model and results are.. surprisingly bad?

Mistral OCR:

- 72.2% accuracy

- $1/1000 pages

- 5.42s / page

Which is pretty far cry from the 95% accuracy they were advertising from their private benchmark. The biggest thing I noticed is how it skips anything it classifies as an image/figure. So charts, infographics, some tables, etc. all get lifted out and returned as [image](image_002). Compared to the other VLMs that are able to interpret those images into a text representation.

https://github.com/getomni-ai/benchmark

https://huggingface.co/datasets/getomni-ai/ocr-benchmark

https://getomni.ai/ocr-benchmark

Re: Mistral OCR

#293

Post by themanmaran »

themanmaran wrote: post #46
ChemSpider wrote: post #27

"World's best OCR model" - that is quite a statement. Are there any well-known benchmarks for OCR software?

We published this benchmark the other week. We'll can update and run with Mistral today! https://github.com/getomni-ai/benchmark

Update: Just ran our benchmark on the Mistral model and results are.. surprisingly bad?

Mistral OCR:

- 72.2% accuracy

- $1/1000 pages

- 5.42s / page

Which is pretty far cry from the 95% accuracy they were advertising from their private benchmark. The biggest thing I noticed is how it skips anything it classifies as an image/figure. So charts, infographics, some tables, etc. all get lifted out and returned as [image](image_002). Compared to the other VLMs that are able to interpret those images into a text representation.

https://github.com/getomni-ai/benchmark

https://huggingface.co/datasets/getomni-ai/ocr-benchmark

https://getomni.ai/ocr-benchmark

Re: Mistral OCR

#294

Post by thelittleone »

mvac wrote: post #142

Great progress, but unfortunately, for our use case (converting medical textbooks from PDF to MD), the results are not as good as those by MinerU/PDF-Extract-Kit [1]. Also the collab link in the article is broken, found a functional one [2] in the docs. [1] https://github.com/opendatalab/MinerU [2] https://colab.research.google.com/github/mistralai/cookbook/...

Have you had a chance to compare results from MinerU vs LLM such a Gemini 2.0 or anthropic's native PDF tool?

Re: Mistral OCR

#295

Post by SilentM68 »

amelius wrote: post #290
SilentM68 wrote: post #55

I would like to see how it performs with massively warped and skewed scanned text images, basically a scanned image where the text lines are wavy as opposed as straight horizontal, where the letters are elongated. One where the line widths are different depending on the position on the scanned image. I once had to deal with such a task that somebody gave me with OCR software, Acrobat, and other tools could not decode…

Are you trying to build a captcha solver?

No, not a captcha solver. When I worked in education, I was given a 90s paper document that a teacher needed OCRd but it was completely warped. It was my job to remediate those type of documents for Accessibility reasons. I had to scan and OCR it but the result was garbage. Mind you I had access to Windows, Linux and MacOS tools but still difficult to do. I had to guess what it said, which was not impossible but it was time-consuming, not doable in the time-frame I was given, so I had no option but to manually retype all the information into a new document and convert it that way. Document remediation and accessibility should be a good use case for A.I., in education.

Re: Mistral OCR

#296

Post by jorvi »

layer8 wrote: post #281
bambax wrote: post #257

It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…

> This is perfect! Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”. These are https://fr.wikipedia.org/wiki/Adverbe_ordinal#Premiers_adver... . There’s also extra spaces after the “1607” and around the hyphen in “Diane-Henriette”. Lastly, U+2019 instead of U+0027 would be more appropriate for…

> Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”

Or degree symbol. Although it should be able to figure out which to use according to the context.

Re: Mistral OCR

#297

Post by renewiltord »

pavl wrote: post #279
renewiltord wrote: post #72

Earlier quoted context omitted.

4o transcribes it perfectly. You can usually root an old Android and write this app in ~2h with LLMs if unfamiliar. The hard part will be maintaining camera lens cleanliness and alignment etc. The time cost is so low that you should give it a gander. You'll be surprised how fast you can do it. If you just take screenshots every minute it should suffice.

What software-tools do you usw to Programm the APP?

Since it's at home, you'll have WiFi access, so it's pretty much a rudimentary Kotlin app on Android. You can just grab a photo and ship it to the GPT-4o API, get the response, and then POST it somewhere.

Re: Mistral OCR

#300

Post by amelius »

cxie wrote: post #40

The new Mistral OCR release looks impressive - 94.89% overall accuracy and significantly better multilingual support than competitors. As someone who's built document processing systems at scale, I'm curious about the real-world implications. Has anyone tried this on specialized domains like medical or legal documents? The benchmarks are promising, but OCR has always faced challenges with domain-specific terminology…

> 94.89% overall accuracy

There are about 47 characters on average in a sentence. So does this mean it gets around 2 or 3 mistakes per sentence?

Post reply on HN