Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited OCR: One-shot long-horizon parsing

github.com

Post reply on HN
Print view View original

31–40 of 119 posts

Re: Unlimited OCR: One-shot long-horizon parsing

#31

Post by Oras »

ljouhet wrote: post #24
Oras wrote: post #2

OCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?

Real question: what tool do you use? (for long/complex documents with tables, code, maths) - marker (with --force-ocr) gives me the best results - Mistral OCR (seems really great, but I never managed to get it work) - Mathpix (tried a long time ago) - docling (gives me garbage, I must use it wrong) - Unlimited OCR (will try it) - ???

- Azure Document Intelligence (has an option to return markdown too including headers and footers).

- AWS Textract

Re: Unlimited OCR: One-shot long-horizon parsing

#32

Post by singpolyma3 »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

What about sheet music typesetting formats like https://abcnotation.com/ ?

Re: Unlimited OCR: One-shot long-horizon parsing

#33

Post by WhitneyLand »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

“there aren't great corpora of training data that would connect a MusicXML representation to sheet music images or to audio”

It may not be necessary…a lot of the training pairs/data for this could probably be procedurally created via code.

Would be pretty fun to work on and see it come to life.

Re: Unlimited OCR: One-shot long-horizon parsing

#34

Post by mcbetz »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

I observe that music OCR space and the only really good solution so far is soundslice. You scan and review some edge cases and get really good results. Paid service by a small company, very worthy to be supported!

Re: Unlimited OCR: One-shot long-horizon parsing

#35

Post by malfist »

chpatrick wrote: post #8
Oras wrote: post #2

OCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?

It absolutely hasn't been solved, it's just got pretty decent in recent years.

Pretty decent might be quiet the stretch. I'd term it almost acceptable, but only if you're using commercial solutions like amazon's textract, doing it with open source tools is at best, extremely painful and vaguely accurate.

Re: Unlimited OCR: One-shot long-horizon parsing

#37

Post by peatmoss »

singpolyma3 wrote: post #32
peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

What about sheet music typesetting formats like https://abcnotation.com/ ?

I forgot to mention ABC. I have seen a few LLMs look at that. There was a model / paper published a couple years back called ChatMusician that built around it.

With the caveat that I'm not terribly fluent in ABC, it seems to me that simple things are simple, but hard things seem to be nearly pathological. And (again, maybe a lapse in my understanding) it seems like there may be a fair number of concepts that are impossible to convey in ABC?

Lastly, if I understand correctly, ABC got its start and is mostly popular as a simplified format for church songbooks. I'd imagine that would, uh, influence the training corpora towards sounding a bit... church songbooky.

EDIT: I may have been overly dismissive of ABC on first glance. It does seem like people have extended it quite a bit, and that it's at least, in theory, capable of encoding most of what I'd expect. And it's human readable, which is a benefit. Though, readability does take a stiff penalty the more richness you add (e.g. dynamics, articulations, stacked notes, etc)

Re: Unlimited OCR: One-shot long-horizon parsing

#38

Post by ai_fry_ur_brain »

d675 wrote: post #30
robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

See, leetcode is useful. As I do this leetcode grind, I’ve been why techniques exist / how they’re used irl. Lots of interesting stuff there

Who said it wasnt useful, dont listen to those people.

Re: Unlimited OCR: One-shot long-horizon parsing

#39

Post by ai_fry_ur_brain »

ljouhet wrote: post #24
Oras wrote: post #2

OCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?

Real question: what tool do you use? (for long/complex documents with tables, code, maths) - marker (with --force-ocr) gives me the best results - Mistral OCR (seems really great, but I never managed to get it work) - Mathpix (tried a long time ago) - docling (gives me garbage, I must use it wrong) - Unlimited OCR (will try it) - ???

poma-ai has really great chunking techniques that chunk the document based on the document structure/heirarchy.

We use it on 200 page IEEE standards that are notoriously complex, filled with tables and diagram. Highly reccomend.

Post reply on HN