Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral OCR

mistral.ai

Post reply on HN
Print view View original

281–290 of 450 posts

Re: Mistral OCR

#281

Post by layer8 »

bambax wrote: post #257

It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…

> This is perfect!

Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”. These are https://fr.wikipedia.org/wiki/Adverbe_ordinal#Premiers_adver....

There’s also extra spaces after the “1607” and around the hyphen in “Diane-Henriette”.

Lastly, U+2019 instead of U+0027 would be more appropriate for the apostrophe, all the more since in the image it looks like the former and not like the latter.

Re: Mistral OCR

#282

Post by PerryStyle »

NalNezumi wrote: post #254
owenpalmer wrote: post #87

This is incredibly exciting. I've been pondering/experimenting on a hobby project that makes reading papers and textbooks easier and more effective. Unfortunately the OCR and figure extraction technology just wasn't there yet. This is a game changer. Specifically, this allows you to associate figure references with the actual figure, which would allow me to build a UI that solves the annoying problem of looking for a…

>a UI that solves the annoying problem of looking for a referenced figure on another page, which breaks up the flow of reading. A tangent but this exact issue is what I was frustrated for a long time with pdf reader and reading science papers. Then I found sioyek that pops up a small window when you hover over links (references and equations and figures) and it solved it. Granted, the pdf file must be in right format…

Zotero's PDF viewer also does this now. Being able to annotate PDFs and having a reference manager has been a life saver.

Re: Mistral OCR

#283

Post by TeMPOraL »

layer8 wrote: post #281
bambax wrote: post #257

It's not bad! But it still hallucinates. Here's an example of an (admittedly difficult) image: https://i.imgur.com/jcwW5AG.jpeg For the blocks in the center, it outputs: > Claude, duc de Saint-Simon, pair et chevalier des ordres, gouverneur de Blaye, Senlis, etc., né le 16 août 1607 , 3 mai 1693 ; ép. 1○, le 26 septembre 1644, Diane - Henriette de Budos de Portes, morte le 2 décembre 1670; 2○, le 17 octobre 1672, Cha…

> This is perfect! Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”. These are https://fr.wikipedia.org/wiki/Adverbe_ordinal#Premiers_adver... . There’s also extra spaces after the “1607” and around the hyphen in “Diane-Henriette”. Lastly, U+2019 instead of U+0027 would be more appropriate for…

This is "reasoning model" stuff even for humans :).

Re: Mistral OCR

#284

Post by daemonologist »

dontlikeyoueith wrote: post #276
techwizrd wrote: post #223

Earlier quoted context omitted.

The challenge I have is how to get bounding boxes for the OCR, for things like redaction/de-identification.

AWS Textract works pretty well for this and is much cheaper than running LLMs.

Textract is more expensive than this (for your first 1M pages per month at least) and significantly more than something like Gemini Flash. I agree it works pretty well though - definitely better than any of the open source pure OCR solutions I've tried.

Re: Mistral OCR

#285

Post by layer8 »

TeMPOraL wrote: post #283
layer8 wrote: post #281

Earlier quoted context omitted.

> This is perfect! Just a nit, but I wouldn’t call it perfect when using U+25CB ○ WHITE CIRCLE instead of what should be U+00BA º MASCULINE ORDINAL INDICATOR, or alternatively a superscript “o”. These are https://fr.wikipedia.org/wiki/Adverbe_ordinal#Premiers_adver... . There’s also extra spaces after the “1607” and around the hyphen in “Diane-Henriette”. Lastly, U+2019 instead of U+0027 would be more appropriate for…

This is "reasoning model" stuff even for humans :).

There is OCR software that analyses which language is used, and then applies heuristics for the recognized language to steer the character recognition in terms of character sequence likelihoods and punctuation rules.

I don’t think you need a reasoning model for that, just better training; although conversely a reasoning model should hopefully notice the errors — though LLM tokenization might still throw a wrench into that.

Re: Mistral OCR

#286

Post by TeMPOraL »

esafak wrote: post #261
risyachka wrote: post #183

Earlier quoted context omitted.

>> Any legacy vendors providing pure OCR are going to get steamrolled by these VLMs. -OR- they can just use these APIs, and considering that they have a client base - which would prefer to not rewrite integrations to get the same result - they can get rid of most code base, replace it with llm api and increase margins by 90% and enjoy good life.

They're going to become commoditized unless they add value elsewhere. Good news for customers.

They are (or at least could easily be) adding value in form of SLA - charging money for giving guarantees on accuracy. This is both better for customer, who gets concrete guarantees and someone to shift liability to, and for the vendor, that can focus on creating techniques and systems for getting that extra % of reliability out of the LLM OCR process.

All of the above are things companies - particularly larger ones - are happy to pay for, because ORC is just a cog in the machine, and this makes it more reliable and predictable.

On top of the above, there are auxiliary value-adds such a vendor could provide - such as, being fully compliant with every EU directive and regulation that's in power, or about to be. There's plenty of those, they overlap, and no one wants to deal with it if they can outsource it to someone who already figured it out.

(And, again, will take the blame for fuckups. Being a liability sink is always a huge value-add, in any industry.)

Re: Mistral OCR

#287

Post by JBiserkov »

simpaticoder wrote: post #211
kiratp wrote: post #166

It's shocking how much our industry fails to see past its own nose. Not a single example on that page is a Purchase Order, Invoice etc. Not a single example shown is relevant to industry at scale.

Another good example would be contracts of any kind. Imagine photographing a contract (like a car loan) and on the spot getting an AI to read it, understand it, forecast scenarious, highlight red flags, and do some comparison shopping for you.

... imagining ...

... hallucinating during read ...

... hallucinating during understand ...

... hallucinating during forecast ...

... highlighting a hallucination as red flag ...

... missing an actual red flag ...

... consuming water to cool myself...

Phew, being an AI is hard!

Re: Mistral OCR

#289

Post by vikp »

I ran a partial benchmark against marker - https://github.com/VikParuchuri/marker .

Across 375 samples with LLM as a judge, mistral scores 4.32, and marker 4.41 . Marker can inference between 20 and 120 pages per second on an H100.

You can see the samples here - https://huggingface.co/datasets/datalab-to/marker_comparison... .

The code for the benchmark is here - https://github.com/VikParuchuri/marker/tree/master/benchmark... . Will run a full benchmark soon.

Mistral OCR is an impressive model, but OCR is a hard problem, and there is a significant risk of hallucinations/missing text with LLMs.

Re: Mistral OCR

#290

Post by amelius »

SilentM68 wrote: post #55

I would like to see how it performs with massively warped and skewed scanned text images, basically a scanned image where the text lines are wavy as opposed as straight horizontal, where the letters are elongated. One where the line widths are different depending on the position on the scanned image. I once had to deal with such a task that somebody gave me with OCR software, Acrobat, and other tools could not decode…

Are you trying to build a captcha solver?
Post reply on HN