121–130 of 143 posts
After paying for Mistral and using it for a while I genuinely hated it. It's a productivity black hole and can't realistically compete with anyone. I chose it only because it was European, but no. I'd rather let my one year subscription go to waste than use anything 'Mistral'.
It's cheap at $4/1k, but I'm hesitant to even benchmark this one again since the previous versions were all "98% accurate based on internal benchmarks of 4 pdfs" and ended up falling short of almost everything else on the market [1]. Even in this one, they just report that OlmOCRBench and OmniDocBench have "known limitations" and that's why they report flagship numbers from their internal benchmark. https://getomni.a…
After paying for Mistral and using it for a while I genuinely hated it. It's a productivity black hole and can't realistically compete with anyone. I chose it only because it was European, but no. I'd rather let my one year subscription go to waste than use anything 'Mistral'.
I assume you mean for coding, for which I would agree. But for OCR Mistral is SOTA :)
Earlier quoted context omitted.
I do not believe this story. Opus 4.8 scanned hundreds of PDFs for me recently with the worst handwriting imaginable. 100% successful, other than one record where even I could not figure out what was written.
I do not believe this story, because of the message I just posted above. That's not really productive lol, I'm glad it worked for you but these models are non-deterministic and 'YMMV' very much applies everywhere. I had it parse receipts (in fairness, in variable lightning), all taken from iPhone cameras in the past year. And yeah, not a great job, about 20% failed to get the date correct. (Not outrageously wrong, e.…
Earlier quoted context omitted.
I assume you mean for coding, for which I would agree. But for OCR Mistral is SOTA :)
I am not totally following this area but the link from a commenter from above suggests that it is not SOTA but on the lower end (but still good): https://getomni.ai/blog/benchmarking-open-source-models-for-...
Earlier quoted context omitted.
I am not totally following this area but the link from a commenter from above suggests that it is not SOTA but on the lower end (but still good): https://getomni.ai/blog/benchmarking-open-source-models-for-...
interesting... I have never seen such a benchmark before, where it forces the model to use a json schema for parsing a document. Sounds a bit counter intuitive to me, but I'm not that deep in the field. I usually just "ask" the model for information that is inside an image pdf. I haven't run any sophisticated benchmarks though...
Earlier quoted context omitted.
interesting... I have never seen such a benchmark before, where it forces the model to use a json schema for parsing a document. Sounds a bit counter intuitive to me, but I'm not that deep in the field. I usually just "ask" the model for information that is inside an image pdf. I haven't run any sophisticated benchmarks though...
No, they don't force the model to use a json schema, they simply use the model to extract the data, and then they feed that OCR result into the pipeline further to evaluate the OCR results against the ground truth, and this is where JSON schema is used, and also another model (gpt4o).
Earlier quoted context omitted.
Opposite advice. It's very useful to me for dev and general tasks. Been using Claude in parallele, it's better not not that much, just 10x (or 100x ?) more expensive.
Sure, well for me it isn't. It has been awful for even toy tasks that opencode's free plan did without an issue. The general sentiment about it is that it is really bad. I wish I knew before paying.
Tested with Malayalam, normal handwriting got accurate but a slight different style got detected as kannada. Have samples if required, which sarvam got done with 99% accuracy leaving one text error.
I’ve always thought the US Postal Service is such a technological marvel. They somehow manage to identify and route billions of pieces of mail and I have to imagine their tech is significantly more primitive than this. Not only that but US addresses are absurdly non-standardized, you can often write the same address multiple ways and have it deliver to the same location. I’m sure there’s plenty of published knowledge…
My father once received a letter from Algeria, with 3 words on the envelope : his first name, "Créteil" (the town where he lived, ≈100k inhabitants), and "France". Of course, in the 70s there was no Internet nor central database to find him, yet the postal service managed to deliver the letter. He was a very active social worker, managed a youth football team, etc. which made him locally well-known by his first name.…