i'm not super familiar with deep learning so forgive me if i'm missing some nuance, but what's the purpose of writing/reading to/from images? seems like it would add a ton of processing time. could the CNN not just read from a 50 item array of tuples representing the data from the 20ms slice?
Finding the genre of a song with Deep Learning
31–36 of 36 posts
Re: Finding the genre of a song with Deep Learning
#32Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…
One reason might be that the mentioned genres are highly formulaic to begin with. The standard rap song contains about 2 bars of unique music stretched out over 3 minutes with slight variations. Same with dubstep and techno. All highly repetitive. Classical music got no drums, so you can detect that. Metal got guitar distortion all over the spectrum. So with these examples the spectral images should have enough disti…
It's quite possible that it's mainly using even more surface-level audio features, before getting to whether the genres are formulaic or not. For example, if specific mastering studios have telltale production features visible in the audio (choice of dynamic range compression algorithms, mixing approaches, etc.), and some mastering studios mainly master, say, country, you can learn to classify country with pretty high accuracy by just recognizing a half-dozen studios' production signatures, without learning anything fundamental about the genre. Whether this happens depends a lot on your choice of data set and validation method.
There's more on that (and some other pitfalls) in a paper linked elsewhere in the comments here: https://news.ycombinator.com/item?id=13085651
Re: Finding the genre of a song with Deep Learning
#33See also http://everynoise.com/ which is a view into how Spotify classifies music.
The creator wrote about it here:
http://blog.echonest.com/post/52385283599/how-we-understand-...
and writes a lot about it on their blog:
http://www.furia.com/page.cgi?terms=noise&type=search
Of course those are going in the other direction, not generating the classification from the data, but it's probably one of the best data sets as far as classifying existing music.
Re: Finding the genre of a song with Deep Learning
#34See also http://everynoise.com/ which is a view into how Spotify classifies music. The creator wrote about it here: http://blog.echonest.com/post/52385283599/how-we-understand-... and writes a lot about it on their blog: http://www.furia.com/page.cgi?terms=noise&type=search Of course those are going in the other direction, not generating the classification from the data, but it's probably one of the best data sets as…
Thanks for the awesome ressources ! :D
Re: Finding the genre of a song with Deep Learning
#35i'm not super familiar with deep learning so forgive me if i'm missing some nuance, but what's the purpose of writing/reading to/from images? seems like it would add a ton of processing time. could the CNN not just read from a 50 item array of tuples representing the data from the 20ms slice?
I'm not sure what you mean, but I have chosen to store slices on the disk so that I could still take a look at them, and not store the data only in numpy arrays. That could be optimize for a better processing time!
Re: Finding the genre of a song with Deep Learning
#36Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…
Yep you got it right, except the voting system adds tons of reliability because we cannot trust the slice classification (2.5s) too much.
I wonder if training another net on top of the slices would work better than voting for a single winner. I'd presume that there are genres that are well characterized by the distribution and progression of their spectrograms. Probably expand/compress the collection of slices to a standard length before training?
(Nice to see you show up for the discussion. I was worried that you'd given up hope before your article hit the front page.)