Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited OCR: One-shot long-horizon parsing

github.com

Post reply on HN
Print view View original

81–90 of 119 posts

Re: Unlimited OCR: One-shot long-horizon parsing

#81

Post by storywatch »

robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

Haven't read the full paper but thr local generation window is a little small, especially since image inputs are especially token heavy. Depending on where the local attention layer is located, it would be nicer if it's bigger e.g. 4096 words at least.

Re: Unlimited OCR: One-shot long-horizon parsing

#82

Post by MattRogish »

robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

I do OCR of images, and that's exactly what I do. I take one big image and slice it into many smaller ones, and send those to the LLM. Perfect every time, unlike using the whole image which resulted in hot garbage.

Re: Unlimited OCR: One-shot long-horizon parsing

#83

Post by elasticdog »

peatmoss wrote: post #71
aidenn0 wrote: post #65

Earlier quoted context omitted.

As someone who has never looked at a jazz score, can you share an example of how jazz sheet music would benefit from different fonts?

It's just an entrenched aesthetic preference. Jazz fonts (fonts in this context refers both to the words and the music symbols) tend to be quite heavy with thick lines. I've heard that the thick hand-written style was originally to make charts more readable in dimly lit clubs, but with tablets and such, that's an anachronism now. You can look at samples of Hal Leonard's Real Book(s) on their website to get a sense of…

I also don't love the conventional handwritten aesthetic you often see for jazz fonts. For a project I've been working on, I ended up pulling the handful of chord symbol glyphs out of MuseScore's Leland Text font and adjusting them for use in the UI since I couldn't find a suitable option out there.

Re: Unlimited OCR: One-shot long-horizon parsing

#84

Post by peterderivaz »

pmarreck wrote: post #23

my attempts at using AI to do OCR have always resulted in invented artifacts, which is not production feasible. does this suffer from that as well? A simple example is words that are supposed to be in other languages being automatically translated to English, which ruins the effect

I've been trying out this model on a 4090 to transcribe a Japanese grammar pdf (written in English with lots of Japanese examples) and it seems to be working very well from the small parts I have double checked. The output contains both the kanji/hiragana and English as appropriate without attempting any translation.

It has converted about 200 pages in an hour.

Re: Unlimited OCR: One-shot long-horizon parsing

#85

Post by freefaler »

MattRogish wrote: post #82
robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

I do OCR of images, and that's exactly what I do. I take one big image and slice it into many smaller ones, and send those to the LLM. Perfect every time, unlike using the whole image which resulted in hot garbage.

It works with relatively good scans, when there are bad/skewed scans and especially something with many label/value pairs, that aren't nicely tucked inside sentences, the more context you have, the more you can find the correct words and fix the errors.

There is a whole class of tricky documents. A decent (if you ignore the marketing bias) post about this problem can be found here:

https://getomni.ai/blog/ocr-benchmark

Re: Unlimited OCR: One-shot long-horizon parsing

#86

Post by dominotw »

timwis wrote: post #79
_puk wrote: post #48

Earlier quoted context omitted.

This hits a sweet spot I think for conversations too. I've been playing (for quite a while) on trying to encapsulate long running conversations. You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc. Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a g…

Can you say more about how this applies to long-running conversations? I've been thinking about them as well, but can't write wrap my head around how this would be better than (or even different to) standard compaction.

standard compactions doesnt really distinguish between long term vs short term ephemeral facts ?

Re: Unlimited OCR: One-shot long-horizon parsing

#87

Post by lacoolj »

This looks more promising than what Mistral just launched (coincidence?????? i think not.)

This approach feels like it could be used for image gen as well (in some combination). Read/view image, start drawing image using illustrator/inkscape/etc (or just SVG), then fill in with what was missed after

Re: Unlimited OCR: One-shot long-horizon parsing

#88

Post by timwis »

dominotw wrote: post #86
timwis wrote: post #79

Earlier quoted context omitted.

Can you say more about how this applies to long-running conversations? I've been thinking about them as well, but can't write wrap my head around how this would be better than (or even different to) standard compaction.

standard compactions doesnt really distinguish between long term vs short term ephemeral facts ?

Forgive me if I'm being naive, but can't you just tweak the compaction prompt to differentiate? Presumably that's what you would do in the separate prompt anyway, right?

Re: Unlimited OCR: One-shot long-horizon parsing

#89

Post by __rito__ »

chpatrick wrote: post #76
malfist wrote: post #35

Earlier quoted context omitted.

Pretty decent might be quiet the stretch. I'd term it almost acceptable, but only if you're using commercial solutions like amazon's textract, doing it with open source tools is at best, extremely painful and vaguely accurate.

PaddleOCR (also from Baidu) is pretty damn good actually.

I have shipped with PaddleOCR to prod. Works pretty well. (Usage limited to printed documents in Anglosphere). Runs fully offline, in CPU.

Re: Unlimited OCR: One-shot long-horizon parsing

#90

Post by ryanisnan »

MattRogish wrote: post #82
robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

I do OCR of images, and that's exactly what I do. I take one big image and slice it into many smaller ones, and send those to the LLM. Perfect every time, unlike using the whole image which resulted in hot garbage.

How do you know where to slice an image? What if you slice an image mid-word?
Post reply on HN