At instances where data accuracy is of paramount importance, i think a hybrid route of non-llm ocr for data parsing and LLMs for structured data extraction is the safe passage to tread on. Seen better results for LLMWhisperer(OCR)[1] and Latest Gemini.
Mistral OCR 3
81–90 of 137 posts
Re: Mistral OCR 3
#82Earlier quoted context omitted.
Right! I didn’t know the difference. Does it mean for 79 out of 100 documents they produce 100% accurate OCR, I doubt it. The win rate sounds like a practical approximation of accuracy here to me. If I am wildly off, I am happy to learn.
79% of the time it beats the previous model. The previous version already achieved up to 99% accuracy in multiple benchmarks, already better than most OCR software.
Thank you.
Re: Mistral OCR 3
#83Earlier quoted context omitted.
>Mistral’s models are pretty good, right Are they? IIRC their best model is still worse than the gpt-oss-120B?
Devstral 2 should be above https://mistral.ai/news/devstral-2-vibe-cli Though I haven't checked other benchmarks and they only report swe
Devstral 2 is free from the API. That has to be a bigger point to what makes it better. The price to performance ratio is practically better in every way.
Does it matter if the performance is slightly worse when it is practically free?
Re: Mistral OCR 3
#84Earlier quoted context omitted.
I've worked on document extraction a lot and while the tweet is too flippant for my taste, it's not wrong. Mistral is comparing itself to non-VLM computer vision services. While not necessarily what everyone needs, they are a very different beasts compared to VLM based extraction because it gives you precise bounding boxes, usually at the cost of larger "document understanding". Its failure mode are also vastly diffe…
Why not use both? I just built a pipeline for document data extraction that uses PaddleOCR, then Gemini 3 to check + fix errors. It gets close to 99.9% on extraction from financial statements finally on par with humans.
I did the opposite. Tesseract to get bboxes, words, and chars and then mistral on the clips with some reasonable reflow to preserve geometry. Paddle wasn’t working on my local machine (until I found RapidOCR). Surya was also very good but because you can’t really tweak any knobs, when it failed it just kinda failed. But Surya > Rapid w/ Paddle > DocTr > Tesseract while the latter gave me the most granularity when I needed it.
Edit: Gemini 2.0 was good enough for VLM cleanup, and now 2.5 or above with structured output make reconstruction even easier.
Re: Mistral OCR 3
#85From a tweet: https://x.com/i/status/2001821298109120856 > can someone help folks at Mistral find more weak baselines to add here? since they can't stomach comparing with SoTA.... > (in case y'all wanna fix it: Chandra, dots.ocr, olmOCR, MinerU, Monkey OCR, and PaddleOCR are a good start)
Also, do you know if their benchmarks are available?
In their website, the benchmarks say “Multilingual (Chinese), Multilingual (East-asian), Multilingual (Eastern europe), Multilingual (English), Multilingual (Western europe), Forms, Handwritten, etc.” However, there’s no reference to the benchmark data.
Re: Mistral OCR 3
#86I'm reading worse performance than many OSS offerings like Paddle, MinerU, MonkeyOCR, etc: https://www.codesota.com/ocr
Their handwriting benchmark is not useful. The test cases aren’t even handwritten!
Re: Mistral OCR 3
#87This might be a good place to check the options available for OCR in-place translations. I took a look at OCR3, but it doesn't seem to support my use-case. It looks more tailored towards data extraction for further processing. I've got some foreign artbooks that I would like to get translated. The translations would need to be in place since the placement of the text relative to the pictures around it is fairly impor…
I’m fairly confident this is solvable quite well with “just two api calls”. Are examples of those books available online?
Re: Mistral OCR 3
#88It seems like Mistral is just chasing around sort of "the fringes" of what could be useful AI features. Are they just getting out-classed by OAI, Google, Anthropic? It seems like EU in general should be heavily invested in Mistral's development, but it doesn't seem like they are.
> It seems like EU in general should be heavily invested in Mistral's development, but it doesn't seem like they are The EU is extremely invested in Mistral's development: half of the effort is finding ways to tax them (hello Zucman tax), the other half is wondering how to regulate them (hello AI act)
Zucman taxes rich individuals (100m€+), not Mistral. AI Act rules are not that difficult to comply with by GPAI model providers as long as the model doesn't become systemic risk... They have to spend a lot more time on PR and handshaking with French politicians than on AI compliance. They probably don't even have a single FTE for that... So that's just prejudice I believe.
Re: Mistral OCR 3
#89[dead]
Re: Mistral OCR 3
#90From a tweet: https://x.com/i/status/2001821298109120856 > can someone help folks at Mistral find more weak baselines to add here? since they can't stomach comparing with SoTA.... > (in case y'all wanna fix it: Chandra, dots.ocr, olmOCR, MinerU, Monkey OCR, and PaddleOCR are a good start)
I've worked on document extraction a lot and while the tweet is too flippant for my taste, it's not wrong. Mistral is comparing itself to non-VLM computer vision services. While not necessarily what everyone needs, they are a very different beasts compared to VLM based extraction because it gives you precise bounding boxes, usually at the cost of larger "document understanding". Its failure mode are also vastly diffe…
Is DeepSeek's not VLM?