Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral OCR

mistral.ai

Post reply on HN
Print view View original

81–90 of 450 posts

Re: Mistral OCR

#81

Post by kapitalx »

Co-founder of doctly.ai here (OCR tool)

I love mistral and what they do. I got really excited about this, but a little disappointed after my first few tests.

I tried a complex table that we use as a first test of any new model, and Mistral OCR decided the entire table should just be extracted as an 'image' and returned this markdown:

``` ![img-0.jpeg](img-0.jpeg) ```

I'll keep testing, but so far, very disappointing :(

This document I try is the entire reason we created Doctly to begin with. We needed an OCR tool for regulatory documents we use and nothing could really give us the right data.

Doctly uses a judge, OCRs a document against multiple LLMs and decides which one to pick. It will continue to run the page until the judge scores above a certain score.

I would have loved to add this into the judge list, but might have to skip it.

Re: Mistral OCR

#82

Post by haswell »

dehrmann wrote: post #52
opwieurposiu wrote: post #25

Related, does anyone know of an app that can read gauges from an image and log the number to influx? I have a solar power meter in my crawlspace, it is inconvenient to go down there. I want to point an old phone at it and log it so I can check it easily. The gauge is digital and looks like this: https://www.pvh2o.com/solarShed/firstPower.jpg

You'll be happier finding a replacement meter that has an interface to monitor it directly or a second meter. An old phone and OCR will be very brittle.

Not OP, but it sounds like the kind of project I’d undertake.

Happiness for me is about exploring the problem within constraints and the satisfaction of building the solution. Brittleness is often of less concern than the fun factor.

And some kinds of brittleness can be managed/solved, which adds to the fun.

Re: Mistral OCR

#83

Post by epolanski »

cxie wrote: post #40

The new Mistral OCR release looks impressive - 94.89% overall accuracy and significantly better multilingual support than competitors. As someone who's built document processing systems at scale, I'm curious about the real-world implications. Has anyone tried this on specialized domains like medical or legal documents? The benchmarks are promising, but OCR has always faced challenges with domain-specific terminology…

At my client we want to provide an AI that can retrieve relevant information from documentation (home building business, documents detail how to install a solar panel or a shower, etc) and we've set up an entire system with benchmarks, agents, etc, yet the bottleneck is OCR!

We have millions and millions of pages of documents and an off by 1 % error means it compounds with the AI's own error, which compounds with documentation itself being incorrect at times, which leads it all to be not production ready (and indeed the project has never been released), not even close.

We simply cannot afford to give our customers incorrect informatiin

We have set up a backoffice app that when users have questions, it sends it to our workers along the response given by our AI application and the person can review it, and ideally correct the ocr output.

Honestly after an year of working it feels like AI right now can only be useful when supervised all the time (such as when coding). Otherwise I just find LLMs still too unreliable besides basic bogus tasks.

Re: Mistral OCR

#84

Post by menaerus »

sunami-ai wrote: post #69

Making Transformers the same cost as CNN's (which are used in character-level ocr, as opposed to image-patch-level) is a good thing. The problem with CNN based character-level OCR is not the recognition models but the detection models. In a former life, I found a way to increase detection accuracy, and, therefore, overall OCR accuracy, and used that as an enhancement on top of Amazon and Google OCR. It worked really…

It certainly isn't the same cost if expressed as a non-subsidized $$$ one needs for the Transformers compute aka infra.

CNNs trained specifically for OCR can run in real time on as small compute as a mobile device is.

Re: Mistral OCR

#85

Post by infecto »

kapitalx wrote: post #81

Co-founder of doctly.ai here (OCR tool) I love mistral and what they do. I got really excited about this, but a little disappointed after my first few tests. I tried a complex table that we use as a first test of any new model, and Mistral OCR decided the entire table should just be extracted as an 'image' and returned this markdown: ``` ![img-0.jpeg](img-0.jpeg) ``` I'll keep testing, but so far, very disappointing…

Why pay more for doctly than an AWS Textract?

Re: Mistral OCR

#86

Post by the_mitsuhiko »

kapitalx wrote: post #81

Co-founder of doctly.ai here (OCR tool) I love mistral and what they do. I got really excited about this, but a little disappointed after my first few tests. I tried a complex table that we use as a first test of any new model, and Mistral OCR decided the entire table should just be extracted as an 'image' and returned this markdown: ``` ![img-0.jpeg](img-0.jpeg) ``` I'll keep testing, but so far, very disappointing…

Would love to see the test file.

Re: Mistral OCR

#87

Post by owenpalmer »

This is incredibly exciting. I've been pondering/experimenting on a hobby project that makes reading papers and textbooks easier and more effective. Unfortunately the OCR and figure extraction technology just wasn't there yet. This is a game changer.

Specifically, this allows you to associate figure references with the actual figure, which would allow me to build a UI that solves the annoying problem of looking for a referenced figure on another page, which breaks up the flow of reading.

It also allows a clean conversion to HTML, so you can add cool functionality like clicking on unfamiliar words for definitions, or inserting LLM generated checkpoint questions to verify understanding. I would like to see if I can automatically integrate Andy Matuschak's Orbit[0] SRS into any PDF.

Lots of potential here.

[0] https://docs.withorbit.com/

Re: Mistral OCR

#89

Post by coolspot »

This is $1 per 1000 pages. For comparison, Azure Document Intelligence is $1.5/1000 pages for general OCR and $30/1000 pages for “custom extraction”.

Re: Mistral OCR

#90

Post by arcfour »

cxie wrote: post #70
unboxingelf wrote: post #47

Earlier quoted context omitted.

We’ll just stick LLM Gateway LLM in front of all the specialized LLMs. MicroLLMs Architecture.

I actually think you're onto something there. The "MicroLLMs Architecture" could mirror how microservices revolutionized web architecture. Instead of one massive model trying to do everything, you'd have specialized models for OCR, code generation, image understanding, etc. Then a "router LLM" would direct queries to the appropriate specialized model and synthesize responses. The efficiency gains could be substantial…

This is already done with agents. Some agents only have tools and the one model, some agents will orchestrate with other LLMs to handle more advanced use cases. It's pretty obvious solution when you think about how to get good performance out of a model on a complex task when useful context length is limited: just run multiple models with their own context and give them a supervisor model—just like how humans organize themselves in real life.
Post reply on HN