171–180 of 252 posts
This is indeed amazing. It's actually how human try to understand and remember things. BY VISUAL! And when memory fade out, the image are getting blurred.
Not sure if those close source multimodal models are already using this method.
Earlier quoted context omitted.
Vision is how humans see text. So text must have built in adaptations to protect from visual noise. For example, two words that look similar must never appear in similar contexts, or else they would be conflated. Hence we can safely reduce such words to the same token. Or something like that.
That also works purely on text and it's the trick I used in my German speech recognition engine ( https://arxiv.org/abs/2206.12693 ). "I'm studying at Oxford Univ" has basically no loss in meaning even though "University" was truncated to less than half its characters.
Earlier quoted context omitted.
Yeah, if it was cross-platform maybe more people would be curious about it, but something that can only run on ~10% of the hardware people have doesn't make it very attractive to even begin to spend time on Apple-exclusive stuff.
10% of hardware is an insanely vast amount, no?
Earlier quoted context omitted.
But this was very much the case with existing OCR software as well? I guess the LLMs will end up making up plausible looking text instead of text riddled with errors, which makes it much harder to catch the mistakes, in fairness
Existing ocr doesn’t skip over entire (legible) paragraphs or hallucinate entire sentences
I find it interesting that there's all these independent AI-OCR Projects but still no commercial offering. Is it still too inaccurate, too complex or simply too expensive?
Earlier quoted context omitted.
Maybe my imagination is limited or our documents aren't complex enough, but are we talking about realistic written documents? I'm sure you can take a screenshot of a very complex spreadsheet and it fails, but in that case you already have the data in structured form anyway, no?
Now if someone mails or faxes you that spreadsheet? You're screwed. Spreadsheets are not the biggest problem though, as they have a reliable 2-dimensional grid - at worst some cells will be combined. The form layouts and n-dimensional table structures you can find on medical and insurance documents are truly unhinged. I've seen documents that I struggled to interpret.
(I imagine it also had the benefit of reducing fraud/errors).
In this day and age, it's probably easier/better to change the process around that as there's little excuse for such shit quality input. I understand this isn't always possible though.
For everyone wondering how good this and other benchmarks are: - the OmniAI benchmark is bad - Instead check OmniDocBench[1] out - Mistral OCR is far far behind most Open Source OCR models and even further behind then Gemini - End to End OCR is still extremely tricky - composed pipelines work better (layout detection -> reading order -> OCR every element) - complex table parsing is still extremely difficult [1]: http…
Wish someone benchmarked Apple Vision Framework against these others. It's built into most Apple devices, but people don't know you can actually harness it to do fast, good quality OCR for you (and go a few extra steps to produce searchable pdfs, which is my typical use case). I'm very curious where it would fall in the benchmarks.
I tried this out on huggingface, and it has the same issue as every other multimodal AI OCR option (including MinerU, olmOCR, Gemini, ChatGPT, ...). It ignores pictures, charts, and other visual elements in a document, even though the models are pretty good at describing images and charts by themselves. What this means is that you can't use these tools yet to create fully accessible alternatives to PDFs.
Earlier quoted context omitted.
10% of hardware is an insanely vast amount, no?
Well, it's 90% less than what everyone else uses, so even if the total number is big, relatively it has a small user-base.
My point is, I don’t think any comparative benchmark would ever exclude something based on “oh it’s just 10%, who cares.” I think the issue is more that Apple Vision Framework is not well known as an OCR option, but maybe it’s starting to change.
And another part of the irony is that Apple’s framework probably gets way more real world usage in practice than most of the tools in that benchmark.
OCR is not a great name for these models. While they can do traditional OCR such as digitize and scanned PDF for example, they do so much more.