Finding the genre of a song with Deep Learning
1–10 of 36 posts
Re: Finding the genre of a song with Deep Learning
#2Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain.
Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samples from the unknown songs, and let it classify them. I find it incredible that 2.5 seconds of sound represented as a tiny picture captures information enough for reliable classification, but apparently it does!
Re: Finding the genre of a song with Deep Learning
#3Nice approach, and well explained! By the way, Niland is a startup that also does music labeling with the help of deep learning.
Demo available here: http://demo.niland.io/
For example, it can output Drum Machine: 87%, House: 88%, Female Voice: 55%, Groovy: 93%
Re: Finding the genre of a song with Deep Learning
#4To the author: Have you tried to use a logarithmic frequency scale in the spectrogram? [1] That representation is closer to the way humans perceive sound, and gives you finer resolution in the lower frequencies. [2] If you want to make your representation even closer to the human's perception, take a look at Google's CARFAC research. [3] Basically, they model the ear. I've prepared a Python utility for converting sound to Neural Activity Pattern (resembles a spectrogram when you plot it) here: https://github.com/iver56/carfac/tree/master/util
[1] https://sourceforge.net/p/sox/feature-requests/176/
Re: Finding the genre of a song with Deep Learning
#5Wow, I find it incredible that this works. As I understand it, the approach is to do a Fourier transform on a couple seconds of the song to create a 128x128 pixel spectrogram. Each horizontal pixel represents a 20 ms slice in time, and each vertical pixel represents 1/128 of the frequency domain. Then treating these spectrograms as images, train a neural net to classify them using pre-labelled samples. Then take samp…
One reason might be that the mentioned genres are highly formulaic to begin with. The standard rap song contains about 2 bars of unique music stretched out over 3 minutes with slight variations. Same with dubstep and techno. All highly repetitive. Classical music got no drums, so you can detect that. Metal got guitar distortion all over the spectrum. So with these examples the spectral images should have enough distinctive features that can be learned. Why should it be different than with 'normal' pictures? Also it looks like they take four 128x128 guesses per song.
Re: Finding the genre of a song with Deep Learning
#6That's pretty cool, I'd like to use something like this to tell me what genre my own songs are. It's annoying to write a song and then upload it to some service or another and have no idea what genre to pick. :-) My stuff is somewhere in the jazz-influenced singer-songwriter american piano pop realm which is a combination that works for me but it generally feels like I'm selling the song short if I have to pick only one.
Re: Finding the genre of a song with Deep Learning
#7See also Bob Sturm's work on genre classification: http://link.springer.com/article/10.1007/s10844-013-0250-y
Re: Finding the genre of a song with Deep Learning
#8Unless I'm misunderstanding the validation set, I'm skeptical of the ability of this classifier to tag unlabeled tracks, given that it is only being trained and tested on tracks which are already known to belong to one of the few trained genres. I'd be curious to see the performance if you were to additionally test on tracks which are not any of (Hardcore, Dubstep, Electro, Classical, Soundtrack and Rap), with a correct prediction being no tag.
Re: Finding the genre of a song with Deep Learning
#9To the author: Have you tried to use a logarithmic frequency scale in the spectrogram? [1] That representation is closer to the way humans perceive sound, and gives you finer resolution in the lower frequencies. [2] If you want to make your representation even closer to the human's perception, take a look at Google's CARFAC research. [3] Basically, they model the ear. I've prepared a Python utility for converting sou…
Mel scale spectrograms are the approach taken in a research paper which uses roughly the same technique as is described in this post: https://dl.dropboxusercontent.com/u/19706734/paper_pt.pdf
Re: Finding the genre of a song with Deep Learning
#10To the author: Have you tried to use a logarithmic frequency scale in the spectrogram? [1] That representation is closer to the way humans perceive sound, and gives you finer resolution in the lower frequencies. [2] If you want to make your representation even closer to the human's perception, take a look at Google's CARFAC research. [3] Basically, they model the ear. I've prepared a Python utility for converting sou…
I don't think this problem is bound by absolute frequency resolution, the tightest distance between two notes on a typical piano is ~2hz and if you assume a doubling between octaves you're at <90 notes. The temporal changes and relative chord progressions probably give more info.