Claude would be good!
claude is live now!
Show HN: OCR Arena – A playground for OCR models
21–30 of 65 posts
Re: Show HN: OCR Arena – A playground for OCR models
#22Really hope there is a layout mode or ocr with bbox mode, I want to see the model restore the whole page.
yeah, that would be a cool long term goal
Re: Show HN: OCR Arena – A playground for OCR models
#23There have been such a large number of OCR tools pop up over the past ~year; sorely in need for some benchmarks to compare them. Would love to see support for normal OCR tools like tesseract, EasyOCR, Microsoft Azure, etc. I'm using these for some projects, and my experiments with VLMs for OCR have resulted in too much hallucination for me to switch. Benchmarks comparing across this aisle would be incredibly useful.
Re: Show HN: OCR Arena – A playground for OCR models
#24A limitation of this leaderboard approach that I want to point out is that while the large general-purpose LLMs can make greater leaps of inference (on handwriting and poor quality scans), and almost always produce better layouts and more coherent output, they can also sometimes be less correct. My experience is that they're more prone to skipping or transposing sections of text, or even hallucinating completely incorrect output, than the purpose-trained models. (A similar comparison can be made in turn to the character- or word-based OCR approaches like Tesseract, which are even less "intelligent" but also even less prone to those malbehaviors.)
Also, some of the models are prone to infinite loops and I suspect this is not being punished appropriately; the frontend seems to get into a bad state after around 50k characters, which prevents the user from selecting a winner. Probably would be beneficial to make sure every model has an output length limit.
Still, a really cool resource - I'm looking forward to more models being added.
Re: Show HN: OCR Arena – A playground for OCR models
#25A limitation of this leaderboard approach that I want to point out is that while the large general-purpose LLMs can make greater leaps of inference (on handwriting and poor quality scans), and almost always produce better layouts and more coherent output, they can also sometimes be less correct . My experience is that they're more prone to skipping or transposing sections of text, or even hallucinating completely inc…
Totally agree w/ your first point! For the looping, we just added a stop condition for now in battle mode, and you can still vote on the other model afterwards. A bit of a hard problem to solve. We will add more models!
Re: Show HN: OCR Arena – A playground for OCR models
#26I've been really impressed with this model specifically because of how insanely cheap it is: https://replicate.com/ibm-granite/granite-vision-3.3-2b I didn't expect IBM to be making relevant AI models but this thing is priced at $1 per 4,000,000 output tokens... I'm using it to transcribe handwritten input text and it works very well and super fast.
Thanks for this! Will test this model out because we do a lot of in between steps to get around the output token limits.
Super nice if it worked for our use case to simply get full output.
Re: Show HN: OCR Arena – A playground for OCR models
#27Would be great to compare these against Apple’s LiveText. This project now supports it:
https://github.com/mkyt/OCRmyPDF-AppleOCR
I’ve had great results locally. Albeit you need macOS >=13 for this.
Re: Show HN: OCR Arena – A playground for OCR models
#28This needs a "both are bad" button. There are some generations where I cannot rightfully beats the other.
Re: Show HN: OCR Arena – A playground for OCR models
#29I suggest you make explicit the assumption that this website is specifically about English text. Otherwise the leaderboard is pretty meaningless, with extreme differences in performance across other scripts - and potentially even languages such as Vietnamese or Czech which use Latin but have lots of accents.
Re: Show HN: OCR Arena – A playground for OCR models
#30Two suggestions:
UX on mobile isn’t great. It wasn’t obvious to me where the second model output was and I was thrown off even more so because the option to vote for model 1 output was presented without ever even seeing model two output.
Second suggestion would be to install a MathJax plugin so one can properly rate mathematical equations and formulas. Raw LATeX is easy to mistake and it makes comparing between LATeX and Unicode outputs hard.