Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited OCR: One-shot long-horizon parsing

github.com

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Unlimited OCR: One-shot long-horizon parsing

#41

Post by peatmoss »

WhitneyLand wrote: post #33
peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

“there aren't great corpora of training data that would connect a MusicXML representation to sheet music images or to audio” It may not be necessary…a lot of the training pairs/data for this could probably be procedurally created via code. Would be pretty fun to work on and see it come to life.

I'd imagine that rendered audio that just used midi voices (even high quality "Real Instruments" midi voices) would be pretty brittle for e.g. stem separation or automatic transcription. In a best case, I think you'd start with a clean digital representation, render sheet music imagery, and then have lots of recordings by a bunch of real instrumentalists playing the same music.

On the topic of stem separation, I've wondered about creating a quasi-synthetic dataset by taking chunks of recordings by real musicians playing them back in a real space in various combinations and recording the resulting analog-blended cacophony. Could repeat in various environments like cathedrals, basement bars, etc for realism :-)

Re: Unlimited OCR: One-shot long-horizon parsing

#42

Post by Xevion »

ai_fry_ur_brain wrote: post #38
d675 wrote: post #30

Earlier quoted context omitted.

See, leetcode is useful. As I do this leetcode grind, I’ve been why techniques exist / how they’re used irl. Lots of interesting stuff there

Who said it wasnt useful, dont listen to those people.

People who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position

Re: Unlimited OCR: One-shot long-horizon parsing

#44

Post by pbhjpbhj »

nickspacek wrote: post #25
gcr wrote: post #20

Earlier quoted context omitted.

I don’t understand the shade being thrown ?

It's the opposite of shade, unless GP is being sarcastic. "Class act" is normally a compliment, and in the context here it sounds to me like they're congratulating Baidu/the researchers in being transparent about where their ideas came from.

To be fair, I think I see "[real] class act" almost always used sarcastically.

Re: Unlimited OCR: One-shot long-horizon parsing

#45

Post by drakmo »

pmarreck wrote: post #23

my attempts at using AI to do OCR have always resulted in invented artifacts, which is not production feasible. does this suffer from that as well? A simple example is words that are supposed to be in other languages being automatically translated to English, which ruins the effect

If I would want to achieve 100% recognition results I would combine this method with an image model recreating the original document from the transcribed text and matching the layout. One can do that with using all but the page or paragraph from the document you want to recreate (to avoid recreating the exact passage under test from the image artifact directly). After reconstructing you can do an optical comparison that specifically matches misaligned characters and find the errors. Rinse and repeat. Expensive but it would guarantee 100% recognition.

Re: Unlimited OCR: One-shot long-horizon parsing

#46

Post by badlibrarian »

Oras wrote: post #31
ljouhet wrote: post #24

Earlier quoted context omitted.

Real question: what tool do you use? (for long/complex documents with tables, code, maths) - marker (with --force-ocr) gives me the best results - Mistral OCR (seems really great, but I never managed to get it work) - Mathpix (tried a long time ago) - docling (gives me garbage, I must use it wrong) - Unlimited OCR (will try it) - ???

- Azure Document Intelligence (has an option to return markdown too including headers and footers). - AWS Textract

Exactly. They're both very expensive and prone to surprising you. Sometimes in a good way, sometimes in a bad way. I'd rate them 85%, but you have to run a test because they both fail in different ways on the 15%.

Re: Unlimited OCR: One-shot long-horizon parsing

#48

Post by _puk »

robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

This hits a sweet spot I think for conversations too. I've been playing (for quite a while) on trying to encapsulate long running conversations.

You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc.

Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a general trend over the longer term.

When trying to reconstruct a conversation you really need to find the right balance without pulling in everything that has ever been discussed.

This definitely is worth further investigation.

Re: Unlimited OCR: One-shot long-horizon parsing

#50

Post by pbhjpbhj »

pmarreck wrote: post #23

my attempts at using AI to do OCR have always resulted in invented artifacts, which is not production feasible. does this suffer from that as well? A simple example is words that are supposed to be in other languages being automatically translated to English, which ruins the effect

You almost don't want [super-]word level ML (ie word-pair/phrase/sentence/document/corpus level).

In transcription, you want near certainty, or you want marking that the word could not be read with certainty - yes, context lets you guess, but you want - for some OCR - to know when it's a guess based on other than the letters in order forming a word.

Example, in a census document on familysearch.com the transcriber "corrected" a name as Joseph. The literal letters in the handwritten document spell Josepth ... and sure enough that's a local variant spelling (Eire).

In another document the writer has used "Joh" as an abbreviation, a [human, I assume] transcriber put that as John ... which is most likely, but happens to be wrong.

Sometimes you care that it's guessed, sometimes you want just the best guess.

Post reply on HN