Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finding the genre of a song with Deep Learning

medium.com

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Finding the genre of a song with Deep Learning

#11

Post by dkarapetyan »

Hmm, convolution is perfectly good operation to run on wave forms as well. In fact the wikipedia article (https://en.wikipedia.org/wiki/Convolution) shows the operation on functions which would correspond to time-domain wave forms. What is the point of converting everything to pictures and then using 2D convolutions when that step could have been skipped entirely?

Converting to pictures is unnecessary. It makes the processing harder. The pooling should just happen on segments of the wave form instead of the fourier transform (frequency-domain) picture spectrograms.

Re: Finding the genre of a song with Deep Learning

#12

Post by highd »

dkarapetyan wrote: post #11

Hmm, convolution is perfectly good operation to run on wave forms as well. In fact the wikipedia article ( https://en.wikipedia.org/wiki/Convolution ) shows the operation on functions which would correspond to time-domain wave forms. What is the point of converting everything to pictures and then using 2D convolutions when that step could have been skipped entirely? Converting to pictures is unnecessary. It makes the…

The idea is that the vertical axis of the spectrogram is basically already an hierarchical set of features (in scale/frequency). Then convolutions on that is a lot like how DenseNets combine hierarchical features.

I agree it seems a little jank, but the features are pretty good - and a lot of network architectures / training techniques are most practiced in an image processing context.

Re: Finding the genre of a song with Deep Learning

#13

Post by dagw »

iammyIP wrote: post #5
nkurz wrote: post #2

Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…

One reason might be that the mentioned genres are highly formulaic to begin with. The standard rap song contains about 2 bars of unique music stretched out over 3 minutes with slight variations. Same with dubstep and techno. All highly repetitive. Classical music got no drums, so you can detect that. Metal got guitar distortion all over the spectrum. So with these examples the spectral images should have enough disti…

If they can write some code that can classify metal into one of its 72 sub-genres then I'll be truly impressed :)

Although I wonder what that would do to the metal scene if their main topic of discussion and contention got completely solved.

Re: Finding the genre of a song with Deep Learning

#15

Post by kimburgess »

nkurz wrote: post #2

Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…

From the description in the walkthrough, it doesn't. The final output looks to be based on 5 of these slices, with each providing a probability distribution that ultimately influences the final classification.

Re: Finding the genre of a song with Deep Learning

#16

Post by Despoisj »

nkurz wrote: post #2

Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…

Yep you got it right, except the voting system adds tons of reliability because we cannot trust the slice classification (2.5s) too much.

Re: Finding the genre of a song with Deep Learning

#17

Post by Despoisj »

kimburgess wrote: post #15
nkurz wrote: post #2

Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…

From the description in the walkthrough, it doesn't. The final output looks to be based on 5 of these slices, with each providing a probability distribution that ultimately influences the final classification.

Sorry if the 5 slices are misleading it was only for readability, the average song has 70 slices, which are all classified and used or voting.

Re: Finding the genre of a song with Deep Learning

#18

Post by Despoisj »

dagw wrote: post #13
iammyIP wrote: post #5

Earlier quoted context omitted.

One reason might be that the mentioned genres are highly formulaic to begin with. The standard rap song contains about 2 bars of unique music stretched out over 3 minutes with slight variations. Same with dubstep and techno. All highly repetitive. Classical music got no drums, so you can detect that. Metal got guitar distortion all over the spectrum. So with these examples the spectral images should have enough disti…

If they can write some code that can classify metal into one of its 72 sub-genres then I'll be truly impressed :) Although I wonder what that would do to the metal scene if their main topic of discussion and contention got completely solved.

Haha that would be awesome ! I guess we'd need a lot of data, and probably use much more detailed spectrogram (time-wise and frequency-wise).

Re: Finding the genre of a song with Deep Learning

#19

Post by Despoisj »

stakecounter wrote: post #8

Unless I'm misunderstanding the validation set, I'm skeptical of the ability of this classifier to tag unlabeled tracks, given that it is only being trained and tested on tracks which are already known to belong to one of the few trained genres. I'd be curious to see the performance if you were to additionally test on tracks which are not any of (Hardcore, Dubstep, Electro, Classical, Soundtrack and Rap), with a corr…

It's true that the validation set only contains genres I used for the training. I'll try this out this evening ;)
Post reply on HN