Live data from Hacker News

Researchers’ AI aligns sheet music with MIDI audio

venturebeat.com

61–68 of 68 posts

Re: Researchers’ AI aligns sheet music with MIDI audio

#61
What I don't get is why the emphasis on sheet music as an image?

I get that mapping from an auditory stream onto a score is an interesting problem, but isn't it naturally decoupled from the visual processing problem of reading and segmenting images of sheet music?

Wouldn't it make more sense to deal primarily with a digital intermediate encoding of the score?

What am I missing?

Re: Researchers’ AI aligns sheet music with MIDI audio

#62
post #6

Earlier quoted context omitted.

What information isn't captured in the MIDI but is captured in the electronic sheet music?

Tons. Nearly everything. Sheet music is very expressive. So is MIDI. It’s kind of like asking, “What information isn’t captured in the movie but is captured in the novel?” They’re just radically different ways of expressing music, and much is lost when you translate from one to the other. If you are curious, try flipping through a book on orchestration at the library sometime, or a book on music theory. Even at its m…

> you can’t tell the difference between G# and Ab

That's a problem with MIDI-the-standard, not MIDI-the-idea. In the 1.x standard, it's not just a simplistic definition; the "known workaround" (pitch bend) for when you actually need to express this difference in a played note, is global per instrument. But it's nothing that can't be fixed with a protocol that follows the same principles, but has a richer data model (like OSC).

> What information isn’t captured in the movie but is captured in the novel?

This, absolutely this. And it also doesn't mean we shouldn't be getting better cameras.

Re: Researchers’ AI aligns sheet music with MIDI audio

#64

Earlier quoted context omitted.

>The score is a reduction that's intentionally lacking tons of information that defines the essence of a performance. This is precisely part of my point, or would be if you corrected it. The score contains, either implicitly or explicitly, global performance clues that will tell the performer what to do about timing (and if it doesn't that's because it is expected to come from a conductor or other similar source). It…

I was remarking on your comment that timing variation is noise, as you're relegating far too much into that category due to your narrow view on what counts as musical intent. Rhythmic variations can be highly irregular and even seemingly random, yet follow an underlying logic and be stylistically essential to the performance, which means they would merit notation in some form. And I'm sorry, but regarding samba, you…

> Rhythmic variations can be highly irregular and even seemingly random, yet follow an underlying logic and be stylistically essential to the performance, which means they would merit notation in some form.

and yet ... in almost all the musical forms where this happens, it isn't notated.

I've played samba (surdo and tamborim parts, mostly). I have many friends who play Brasilian music in general. I think we have a problem with definitions, because the pandeiro part precisely fits my definition of "extremely regular timing". When playing samba, unlike various jazz influenced forms, you do not play ahead or behind the groove. The variation still uses a 16th note grid, albeit with lots of freedom of which parts of the grid to play or not play.

Re: Researchers’ AI aligns sheet music with MIDI audio

#65

Earlier quoted context omitted.

>The score is a reduction that's intentionally lacking tons of information that defines the essence of a performance. This is precisely part of my point, or would be if you corrected it. The score contains, either implicitly or explicitly, global performance clues that will tell the performer what to do about timing (and if it doesn't that's because it is expected to come from a conductor or other similar source). It…

> Samba is an extremely regular groove that is very easy to understand. This is true of most Afro-Cuban derived rhythmic structures Samba isn’t Afro-Cuban derived, whether in its rhythmic structures or otherwise.

Sorry, I wrote that down in a misleading way. Afro/Cuban might have been better, but what I really meant was "rhythmic structures influenced by African and/or Cuban musical culture", which samba definitely was. That's redundant in a way, because Cuban forms themselves are deeply indebted (if not directly playing) African forms, but my understanding is that there was a particular development/expansion of things in Cuba (much as in Brasil) that is perhaps best understood as a new branch of the tree. Samba doesn't owe anything in particular to Cuban traditions, but does draw on other African percussive structures.

Re: Researchers’ AI aligns sheet music with MIDI audio

#66

Earlier quoted context omitted.

I was remarking on your comment that timing variation is noise, as you're relegating far too much into that category due to your narrow view on what counts as musical intent. Rhythmic variations can be highly irregular and even seemingly random, yet follow an underlying logic and be stylistically essential to the performance, which means they would merit notation in some form. And I'm sorry, but regarding samba, you…

> Rhythmic variations can be highly irregular and even seemingly random, yet follow an underlying logic and be stylistically essential to the performance, which means they would merit notation in some form. and yet ... in almost all the musical forms where this happens, it isn't notated. I've played samba (surdo and tamborim parts, mostly). I have many friends who play Brasilian music in general. I think we have a pr…

I see. I took "extremely regular" to mean conforming to an equidistant grid structure, while you seem to have been talking about the consistency in repetition that makes it a groove.

Regardless, the context of this discussion was determining whether MIDI data can hold "extra information" that the score does not, and I can't agree with most of your statements about that.

> technically, that would be a case of the MIDI data having noise in it, not extra information.

> and yet ... in almost all the musical forms where this happens, it isn't notated.

There's a term for this type of thinking, and it's called "notational centricity."

> "musicological methods tend to foreground those musical parameters which can be easily notated" such as pitch relationships or the relationship between words and music. On the other hand, historical musicology tends to "neglect or have difficulty with parameters which are not easily notated", such as tone colour or non-Western rhythms.

So any given parameter not being notated with ease or in detail doesn't prove anything about its role as an intentional, stylistic element of the music. That is, not being notated doesn't make a parameter any more likely to be "noise," because what does get notated is not a "core representation" of the musical text. In fact, the distinction between the two mostly comes down to historical coincidence or other non-musical factors.

MIDI or other more granulous performance capture standards have plenty of "extra information" to offer that is not noise, and I'd even say it's mostly the case that you'll see a robust SNR there.

Re: Researchers’ AI aligns sheet music with MIDI audio

#67
post #53
post #44

Earlier quoted context omitted.

Sheet music has far less structure than you think it does.

Can you explain what you mean by that? Sheet music would seem to be incredibly structured to me. Unless we’re talking about sheet music hand scribbled on paper?

I'm guessing you've never taken an academic instrumentation or orchestration course. Sheet music is like HL7, a bunch of common practices that at a passing glance look like a standard but have enough exceptions and idiomatic variance to drive you to drink.

Re: Researchers’ AI aligns sheet music with MIDI audio

#68
post #16
post #2

If you already have a MIDI why would you need the sheet music?

The first example I can think of, because I run into it often, is when you use syncopation. It's made even worse if the syncopated line contains chords, and then even more-so if the note lengths within the chord are not robotically static. Writing that on the MIDI piano roll is slobber proof. But if you take music written that way and view it as sheet music, it is nearly impossible to read because of the many ties th…

Does your midi software properly translate drum rudiments that have specific notation symbols? That would be quite impressive actually. Drumming has a lot of technique-specific and informal stylistic things that would be hard to translate.
Post reply on HN