Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral OCR

mistral.ai

Post reply on HN
Print view View original

261–270 of 450 posts

Re: Mistral OCR

#261

Post by esafak »

risyachka wrote: post #183
kbyatnal wrote: post #140

We're approaching the point where OCR becomes "solved" — very exciting! Any legacy vendors providing pure OCR are going to get steamrolled by these VLMs. However IMO, there's still a large gap for businesses in going from raw OCR outputs —> document processing deployed in prod for mission-critical use cases. LLMs and VLMs aren't magic, and anyone who goes in expecting 100% automation is in for a surprise. You still n…

>> Any legacy vendors providing pure OCR are going to get steamrolled by these VLMs. -OR- they can just use these APIs, and considering that they have a client base - which would prefer to not rewrite integrations to get the same result - they can get rid of most code base, replace it with llm api and increase margins by 90% and enjoy good life.

They're going to become commoditized unless they add value elsewhere. Good news for customers.

Re: Mistral OCR

#262

Post by bambax »

bambax wrote: post #257

It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…

Another test with a text in English, which is maybe more fair (although Mistral is a French company ;-). This image is from Parliamentary debates of the parliament of New Zealand in 1854-55:

https://i.imgur.com/1uVAWx9.png

Here's the output of the first paragraph, with mistakes in brackets:

> drafts would be laid on the table, and a long discussion would ensue; whereas a Committee would be able to frame a document which, with perhaps a few verbal emundations [emendations], would be adopted; the time of the House would thus be saved, and its business expected [expedited]. With regard to the question of the comparative advantages of The-day [Tuesday]* and Friday, he should vote for the amendment, on the principle that the wishes of members from a distance should be considered on all sensations [occasions] where a principle would not be compromised or the convenience of the House interfered with. He hoped the honourable member for the Town of Christchurch would adopt the suggestion he (Mr. Forssith [Forsaith]) had thrown out and said [add] to his motion the names of a Committee.*

Some mistakes are minor (emnundations/emendations or Forssith/Forsaith), but others are very bad, because they are unpredictable and don't correspond to any pattern, and therefore can be very hard to spot: sensations instead of occasions, or expected in lieu of expedited... That last one really changes the meaning of the sentence.

Re: Mistral OCR

#263

Post by albatrosstrophy »

evmar wrote: post #129

I noticed on the Arabic example they lost a space after the first letter on the third to last line, can any native speakers confirm? (I only know enough Arabic to ask dumb questions like this, curious to learn more.) Edit: it looks like they also added a vowel mark not present in the input on the line immediately after. Edit2: here's a picture of what I'm talking about, the before/after: https://ibb.co/v6xcPMHv

And here I thought after reading the headline: finally a reliable Arabic OCR. I've never in my life found a good that does the job decently especially for a scanned document. Or is there something out there I don't know about?

Re: Mistral OCR

#264

Post by pilooch »

simonw wrote: post #186
pilooch wrote: post #158

But what's the need exactly for OCR when you have multimodal LLMs that can read the same info and directly answer any questions about it ? For a VLLM, my understanding is that OCR corresponds to a sub-field of questions, of the type 'read exactly what's written in this document'.

The biggest risk of vision LLMs for OCR is that they might accidentally follow instructions is the text that they are meant to be processing. (I asked Mistral if their OCR system was vulnerable to this and they said "should be robust, but curious to see if you find any fun examples" - https://twitter.com/simonw/status/1897713755741368434 and https://twitter.com/sophiamyang/status/1897719199595720722 )

Fun, but LLMs would follow them post OCR anyways ;)

I see OCR much like phonemes in speech, once you have end to end systems, they become latent constructs from the past.

And that is actually good, more code going into models instead.

Re: Mistral OCR

#265

Post by riquito »

Thaxll wrote: post #157
riquito wrote: post #143

Earlier quoted context omitted.

It fits the "use cases" mentioned in the article > Preserving historical and cultural heritage: Organizations and nonprofits that are custodians of heritage have been using Mistral OCR to digitize historical documents and artifacts, ensuring their preservation and making them accessible to a broader audience.

There is a difference between historical document and "my doctor prescription". Someone coming here and saying it does not work with my old german hanwriting doesn't say much.

You're making a strawman, the parent specifically mentioned "historical handwritten documents"

Re: Mistral OCR

#266

Post by spudlyo »

bambax wrote: post #257

It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…

I want to rejoice that OCR is now a "solved" problem, but I feel like hallucinations are just as problematic as the kind of stuff I have to put up with tesseract -- both require careful manual proofreading for an acceptable degree of confidence. I guess I'll have to try it and see for myself just how much better these solutions are for my public domain archive.org Latin language reader & textbook projects.

Re: Mistral OCR

#268

Post by bugglebeetle »

Squarex wrote: post #148
bugglebeetle wrote: post #105

Congrats to Mistral for yet again releasing another closed source thing that costs more than running an open source equivalent: https://github.com/DS4SD/docling

I am all for open source, but where do you see benchmarks that conclude that it's just equivalent?

Where do you see open source benchmark results that confirm Mistral’s performance?

Re: Mistral OCR

#269

Post by kiratp »

merb wrote: post #215
kiratp wrote: post #166

It's shocking how much our industry fails to see past its own nose. Not a single example on that page is a Purchase Order, Invoice etc. Not a single example shown is relevant to industry at scale.

Mistral is Europe based where invoices are more or less sent digitally in like 95% of all the cases anyway. Some are even digital invoices, which will at some point in the eu be mandatory. For orders there are proposals for that, too. And basically invoice data extraction is a different beast.

This isn't even close to true.

Source: We have large EU customers.

Re: Mistral OCR

#270

Post by yoda97 »

gl-prod wrote: post #175
evmar wrote: post #168

Earlier quoted context omitted.

I am pretty sure it added a kasrah not present in the input on the 2nd to last line. (Not saying it's not super impressive, and also that almost certainly is the right word, but I think that still means not quite "perfect"?)

Yes, it looks like it did add a kasrah to the word ظهري

Yep, and فمِنا too, this is not just OCR, it made some post-processing corrections or "enhancements". That could be good, but it could also be trouble the 1% chance it makes a mistake in critical documents.
Post reply on HN