TuneNN: A transformer-based network model for pitch detection • Hacker News Board

Re: TuneNN: A transformer-based network model for pitch detection

#11

Post by advisedwang » Tue, Dec 19, 2023, 5:50 PM UTC

azinman2 wrote: post #6

CMLab wrote: post #2

A transformer-based network model, pitch tracking for musical instruments. The timbre of musical notes is the result of various combinations and transformations of harmonic relationships, harmonic strengths and weaknesses, instrument resonant peaks, and structural resonant peaks over time. It utilizes the transformer-based tuneNN network model for abstract timbre modeling, supporting tuning for 12+ instrument types.

This smells like an automated summary.

It is from the submitter. I think it is intended as a submission statement.

Re: TuneNN: A transformer-based network model for pitch detection

#12

Post by CMLab » Tue, Dec 19, 2023, 6:05 PM UTC

rrherr wrote: post #8

How does the accuracy of this compare to CREPE? https://github.com/marl/crepe https://github.com/maxrmorrison/torchcrepe Does anyone know what the current state of the art is, within the Music Information Retrieval community?

CREPE generally has high latency and error rates in instrument pitch recognition, especially for guitar instruments. Our team will release benchmark test data and results later.

Re: TuneNN: A transformer-based network model for pitch detection

#13

Post by CMLab » Tue, Dec 19, 2023, 6:08 PM UTC

ranting-moth wrote: post #10

To the dev: the tuner gives me an incredibly high error window with the following message. It doesn't prompt to access the mic (I think that's related). Ubuntu/KDE/Firefox: An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was: TypeError: 'microphone' (value of 'name' member of PermissionDescriptor) is not a valid value for enumeration PermissionNam…

Thank you for providing error feedback. We will work hard to address it. Currently, the model-related data is relatively large, which may be related to network speed.

Re: TuneNN: A transformer-based network model for pitch detection

#14

Post by CMLab » Tue, Dec 19, 2023, 6:11 PM UTC

vessenes wrote: post #3

This is cool! The very best software-based tuning tech out there is probably in piano tuning apps; they cost hundreds of dollars+ and are specifically made to report on harmonics and other piano nuances. Do you have any comparisons against other pitch detection tech? Accuracy? Delay/Responsiveness? I assume it's much more compute work than a handcoded FFT type pitch detector. I think it's possible this would find uti…

Based on our current tests, our algorithm shows significantly higher accuracy and robustness compared to traditional digital signal algorithms such as PEF, NCF, YIN, HPS, etc. Our team is working diligently, and we will release benchmark test data and results in the near future.

Re: TuneNN: A transformer-based network model for pitch detection

#15

Post by im3w1l » Tue, Dec 19, 2023, 6:31 PM UTC

CMLab wrote: post #14

vessenes wrote: post #3

This is cool! The very best software-based tuning tech out there is probably in piano tuning apps; they cost hundreds of dollars+ and are specifically made to report on harmonics and other piano nuances. Do you have any comparisons against other pitch detection tech? Accuracy? Delay/Responsiveness? I assume it's much more compute work than a handcoded FFT type pitch detector. I think it's possible this would find uti…

Based on our current tests, our algorithm shows significantly higher accuracy and robustness compared to traditional digital signal algorithms such as PEF, NCF, YIN, HPS, etc. Our team is working diligently, and we will release benchmark test data and results in the near future.

That's pretty nice. Do you have any idea how it does it?

Re: TuneNN: A transformer-based network model for pitch detection

#16

Post by ska » Tue, Dec 19, 2023, 6:37 PM UTC

jansommer wrote: post #9

ronsor wrote: post #5

Earlier quoted context omitted.

Have you also considered implementing the Nebula[1] algorithm? [1] https://github.com/Sleepwalking/nebula

I need non-GPL libraries as a reference. The problem with YIN and especially PYIN is that the MIT-code I've found sometimes looks a bit too similar to earlier code in GPL. Rewriting that into the same but in different code is fairly hard. Here I'm assuming that translating eg. GPL Python or C++ into C would mean the license is retained

Can you not just write it from the paper(s)? Or is that more effort than value to you?

> that translating eg. GPL Python or C++ into C would mean the license is retained

It depends a bit on what exactly "translating" means but you could easily be a derivative work.

Honestly in that situation I wouldn't even look at the code. You might use in to test equivalent behavior after you have your own implementation, but only in a gross sense.

Re: TuneNN: A transformer-based network model for pitch detection

#17

Post by rrherr » Tue, Dec 19, 2023, 6:41 PM UTC

CMLab wrote: post #12

rrherr wrote: post #8

How does the accuracy of this compare to CREPE? https://github.com/marl/crepe https://github.com/maxrmorrison/torchcrepe Does anyone know what the current state of the art is, within the Music Information Retrieval community?

CREPE generally has high latency and error rates in instrument pitch recognition, especially for guitar instruments. Our team will release benchmark test data and results later.

Thanks. I’d love to try TuneNN! Are you releasing a pretrained model? How do I run it on a wav file?

Re: TuneNN: A transformer-based network model for pitch detection

#18

Post by azinman2 » Tue, Dec 19, 2023, 6:46 PM UTC

advisedwang wrote: post #11

azinman2 wrote: post #6

Earlier quoted context omitted.

This smells like an automated summary.

It is from the submitter. I think it is intended as a submission statement.

That’s the first time I’ve seen/noticed that. I think it makes me feel better?

Re: TuneNN: A transformer-based network model for pitch detection

#19

Post by jansommer » Tue, Dec 19, 2023, 6:53 PM UTC

ska wrote: post #16

jansommer wrote: post #9

Earlier quoted context omitted.

I need non-GPL libraries as a reference. The problem with YIN and especially PYIN is that the MIT-code I've found sometimes looks a bit too similar to earlier code in GPL. Rewriting that into the same but in different code is fairly hard. Here I'm assuming that translating eg. GPL Python or C++ into C would mean the license is retained

Can you not just write it from the paper(s)? Or is that more effort than value to you? > that translating eg. GPL Python or C++ into C would mean the license is retained It depends a bit on what exactly "translating" means but you could easily be a derivative work. Honestly in that situation I wouldn't even look at the code. You might use in to test equivalent behavior after you have your own implementation, but only…

I think I have to look at the code when using other people's MIT licensed code... If they have used something that's GPL or used someone else's code that turns out to be GPL, then it becomes my problem when translating it. And I'm not smart enough to just follow a paper

Re: TuneNN: A transformer-based network model for pitch detection

#20

Post by joonatan » Tue, Dec 19, 2023, 6:58 PM UTC

Could someone fill me in why would machine learning be necessary for pitch detection? Isn't it something that could just be solved with FFT or it's a much more complicated task?