Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unlimited OCR: One-shot long-horizon parsing

github.com

Post reply on HN
Print view View original

51–60 of 119 posts

Re: Unlimited OCR: One-shot long-horizon parsing

#51

Post by genxy »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

Create a benchmark for this problem that researchers can easily run and the problem will solve itself.

Re: Unlimited OCR: One-shot long-horizon parsing

#53

Post by ewild »

_puk wrote: post #48
robotswantdata wrote: post #5

Very interesting. The way I understand this works is that the researchers found a clever architectural hack to stop AI from hoarding memory when reading long documents. Normally, when an AI transcribes a 100 page PDF, it tries to remember every single word it has already ingested. This short-term memory (the KV cache) grows linearly O(N) until the model runs out of VRAM and crashes (or caps it) To avoid this, develop…

This hits a sweet spot I think for conversations too. I've been playing (for quite a while) on trying to encapsulate long running conversations. You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc. Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a g…

This sounds like we are trying to add an LSTM into a transformer

Re: Unlimited OCR: One-shot long-horizon parsing

#54

Post by galbar »

Xevion wrote: post #42
ai_fry_ur_brain wrote: post #38

Earlier quoted context omitted.

Who said it wasnt useful, dont listen to those people.

People who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position

As someone that gets very annoyed when having to do LeetCode in interviews...

Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE. I've had teammates that didn't understand them and everything was fine until when it wasn't (scaling and performance issues).

Or, as I put it to a teammate: "Would you rather review the PR of someone that understands the difference between a set and a list or the PR of someone who doesn't?". This was after we interviewed a candidate with ~15 YoE, on paper, that didn't know the difference.

Re: Unlimited OCR: One-shot long-horizon parsing

#55

Post by indiv0 »

peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

> music is basically a greenfield for AI wherever you look

AIN'T THAT THE TRUTH.

My girlfriend is studying musicology and she has some physical disabilities that make it difficult for her to write things down sometimes. So I try to help her by writing some AI-powered TTS/OCR/etc. apps here and there. It becomes painfully obvious that music was never considered an important part of any AI training dataset, anywhere.

These days, I'm pleasantly surprised by how well Opus 4.8 understands/explains music theory (as you said). But ask him to transcribe/OCR/OMR some sheet music and he'll confidently give you the MusicXML/Lilypond equivalent of "2 + 2 = horse".

I really hope this ignored area will be swept up with the rest of the rising AI wave, but it's still criminally undervalued.

Re: Unlimited OCR: One-shot long-horizon parsing

#56

Post by peatmoss »

mcbetz wrote: post #34
peatmoss wrote: post #29

I recently bought a tablet for sheet music, mostly to replace a stack of jazz "Real Books" at jam sessions. And the phone camera scans I made are okay, but fixed in size and have a lot of artifacts. And it would be great to transpose on the fly for e.g. Bb or Eb instruments, but being a scan this is obviously not possible. I got digging into the state of optical music recognition and came away concluding that music i…

I observe that music OCR space and the only really good solution so far is soundslice. You scan and review some edge cases and get really good results. Paid service by a small company, very worthy to be supported!

I just signed up a trial, and uploaded a messy Real Book scan. It did very well! It missed the coda markings, but then again the directive in the Real Book was nonstandard. I guess that's a case where a multimodal model might have been able to read the text ("after solos, D.C. al coda") and do something smarter.

Re: Unlimited OCR: One-shot long-horizon parsing

#58

Post by elliottcarlson »

galbar wrote: post #54
Xevion wrote: post #42

Earlier quoted context omitted.

People who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position

As someone that gets very annoyed when having to do LeetCode in interviews... Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE. I've had teammates that didn't understand them and everything was fine until when it wasn't (scaling and performance issues). Or, as I put it to a teammate: "Would you rather review the PR of someone that understands the difference between a…

> Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE

Agree with this; however knowing how to roll your own BFS/LRU/etc isn't -- in that case I'd rather review the PR of someone who understands how to leverage tested and known implementations than the PR of someone who decided to roll their own.

Post reply on HN