Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited OCR: One-shot long-horizon parsing

github.com

Post reply on HN
Print view View original

71–80 of 119 posts

Re: Unlimited OCR: One-shot long-horizon parsing

#71

Post by peatmoss »

aidenn0 wrote: post #65
peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

As someone who has never looked at a jazz score, can you share an example of how jazz sheet music would benefit from different fonts?

It's just an entrenched aesthetic preference. Jazz fonts (fonts in this context refers both to the words and the music symbols) tend to be quite heavy with thick lines. I've heard that the thick hand-written style was originally to make charts more readable in dimly lit clubs, but with tablets and such, that's an anachronism now.

You can look at samples of Hal Leonard's Real Book(s) on their website to get a sense of what it looks like. Again, just an aesthetic preference, but one I and many others hold nonetheless.

Re: Unlimited OCR: One-shot long-horizon parsing

#72

Post by mamcx »

Oras wrote: post #2

OCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?

Aside: what is the best to read receipts/bank statements/invoices?

Re: Unlimited OCR: One-shot long-horizon parsing

#74

Post by ai_fry_ur_brain »

Xevion wrote: post #42
ai_fry_ur_brain wrote: post #38

Earlier quoted context omitted.

Who said it wasnt useful, dont listen to those people.

People who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position

Who care's if the leetcode question doesn't relate to the job itself, it shows whether or not the person is willing to put in the work and gives you a glimpse into their ability to reason about hard problems.

Re: Unlimited OCR: One-shot long-horizon parsing

#75

Post by peatmoss »

kwon-young wrote: post #64
peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

So, the format for musicologist and researcher in music is the MEI format: https://music-encoding.org/ for which the reference engraver is verovio: https://www.verovio.org/index.xhtml Note that verovio is able to engrave in svg format while keeping a maximum of information from the original mei score, meaning that you can extract enough metadata to create an actual detection dataset for a deep learning model. This is…

Thank you for this! Both MEI format and the Verovio engraver are news to me. I will check them out.

My first thought was whether MEI format is being added to MuseScore (the sheet music editor I use these days). It looks like it is: https://music-encoding.org/musescore-doc/

As a somewhat related aside, now that the MuseScore people own Hal Leonard and seem to pushing integration with their cloud subscription service, I wonder if they'll see some of these directions as potentially competing with them. I don't think there's anyone who wouldn't love a transposable clean digital version of their Real Books... and if Hal Leonard is in the business of selling Real Books, I can see where good OMR might be a problem for them. I guess piracy of scanned versions is already rampant, so maybe it's a wash.

Re: Unlimited OCR: One-shot long-horizon parsing

#76

Post by chpatrick »

malfist wrote: post #35
chpatrick wrote: post #8

Earlier quoted context omitted.

It absolutely hasn't been solved, it's just got pretty decent in recent years.

Pretty decent might be quiet the stretch. I'd term it almost acceptable, but only if you're using commercial solutions like amazon's textract, doing it with open source tools is at best, extremely painful and vaguely accurate.

PaddleOCR (also from Baidu) is pretty damn good actually.

Re: Unlimited OCR: One-shot long-horizon parsing

#77

Post by SyneRyder »

gettingoverit wrote: post #62
Oras wrote: post #2

OCR has been solved long time ago with vision models. Solutions are consistent, reliable, and stable. What is the point of reinventing the wheel? I would definitely understand post processing, like extracting data, answering question .. etc, but why re-doing the OCR engine itself?

Is it? I've never seen a single OCR that would replace a human just typing it by hand. What if the goal is something actually useful, such as converting scientific paper PDF back to LaTeX that renders into a pixel-perfect copy? What about converting tables from electronics datasheets into computer-readable form? I wouldn't even expect it in the next decade.

I've had success with vision models & OCR, saved me many hours / days / weeks of typing work.

Last year I finally OCR'd many hundreds of pages of my father's old writings. I found that feeding it to Claude Sonnet 4.x via API gave me results that were perfect. No corrections required. So perfect, that Claude was reading along with the story, and actually pointed out a continuity error in the story where an incorrect character was reciting dialog. Claude asked if it should transcribe exactly as is or if I would like Claude to correct the continuity error.

Claude also correctly OCR'd some handwriting that was in the margins of the documents. Sonnet came very close to transcribing a Word Sleuth puzzle, but that was where I hit the limits of its capability at the time.

Mistral OCR was also good (and actually what I started with), but it wasn't quite as good as Claude. And when it was wrong, Mistral could be frighteningly wrong - one API call must have failed, the model must have been presented with a pure black / null image, and I got back a "transcription" that described neverending darkness. It read like something the Woodsman would have broadcast in Twin Peaks S3E8. That poor model.

Tables from electronics datasheets might be okay, I think I've had success with OCR of technical manuals with tables for 80s synthesizer hardware. But I admit my use cases don't crossover into transcriptions of equations or graphs.

Re: Unlimited OCR: One-shot long-horizon parsing

#78

Post by functional_dev »

piterrro wrote: post #52

can someone explain how is this different than feeding the VLM model one page at a time?

I did not know this until today... vision models don't hunt for the white space between letters like old OCR does. They just chop the whole image into a fixed grid of equal squares (patches) and treat each square like a word in a sentence.

This will help if you want to dig deeper - https://vectree.io/c/how-ocr-works-traditional-pipelines-vs-...

Re: Unlimited OCR: One-shot long-horizon parsing

#79

Post by timwis »

_puk wrote: post #48
robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

This hits a sweet spot I think for conversations too. I've been playing (for quite a while) on trying to encapsulate long running conversations. You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc. Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a g…

Can you say more about how this applies to long-running conversations? I've been thinking about them as well, but can't write wrap my head around how this would be better than (or even different to) standard compaction.

Re: Unlimited OCR: One-shot long-horizon parsing

#80

Post by elasticdog »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

For just chord analysis, there's "Harte notation", which is meant to be unambiguous representation of the notes (https://ismir2005.ismir.net/proceedings/1080.pdf). That obviously doesn't get you all of the additional information necessary for engraving and full representation of the music, but there are research datasets available using it like https://github.com/smashub/choco. I've also used the https://github.com/MarkGotham/When-in-Rome dataset for some analysis work, but again that's not 100% what you're looking for.

You might like the "iReal Pro" app for the replacement and transposition of jazz standards on your tablet. It's pretty great for that use case versus camera scans.

Post reply on HN