Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Applying machine learning and deep learning methods to audio analysis

comet.ml

Post reply on HN
Print view View original

11–20 of 21 posts

Re: Applying machine learning and deep learning methods to audio analysis

#11

Post by jononor »

syntaxing wrote: post #2

Is there a method to detect a specific word and tell me the timestamp throughout an audio sample easily? I've been trying to implement something like this but wasn't sure how to approach it.

If you can utilize a cloud API, speech transcription route is likely the simplest. Recognizing spoken words is challenging and data-demanding when it can be spoken by many different speakers.

But if you want to do this on the audio you chop up your audio stream into fixed-length (in time) analysis windows. These length of the window should be a bit longer than the sound of interest (the word). Overlap is normally used for the windows. Say with 90% overlap the next window is created by moving forward by 10%. This gives the model multiple "shots" at detecting the word as it passes by. This is suitable for spotting a word and giving the time within something like 50ms resolution.

For each analysis window you apply feature pre-processing and a model such as the one shown in the article.

This task sounds like what is called Keyword Spotting in academic literature. Which can be seen as as specific version of Audio Event Detection, applied to spoken words.

Re: Applying machine learning and deep learning methods to audio analysis

#12

Post by jononor »

gidim wrote: post #10
jononor wrote: post #6

As an introduction introduction I guess this is OK. However there are two major limitations: 1: The feature extraction ends with mean-summarizing across the entire audio clip - leaving no temporal information. This only works well for simple tasks. At least mentioning something about analysis windows and temporal modelling would be good, as the natural next step. Be it LSTM/GRU on the MFCC, or CNN on mel-spectrogram.…

we're working on another version fixing the folds issue on Urbansound8k and will update the article asap.

Nice!

Re: Applying machine learning and deep learning methods to audio analysis

#13

Post by gidim »

jononor wrote: post #12
gidim wrote: post #10

Earlier quoted context omitted.

we're working on another version fixing the folds issue on Urbansound8k and will update the article asap.

Nice!

just to clarify - are you referring to this experiment? https://www.comet.ml/demo/urbansound8k/be09e32700cd435fb6b55...

Re: Applying machine learning and deep learning methods to audio analysis

#14

Post by jononor »

gidim wrote: post #13
jononor wrote: post #12

Earlier quoted context omitted.

Nice!

just to clarify - are you referring to this experiment? https://www.comet.ml/demo/urbansound8k/be09e32700cd435fb6b55...

Sure, that demonstrates the issue. Problem is with using train_test_split(X, yy, test_size=0.2..) - this assumes independent samples, which is violated for this dataset (because some come from same source audio files). The easiest (and completely acceptable) is to use one fold as the validation data, one fold for the test set, and the remaining folds as training.

This problem is unfortunately quite common even in academic papers using this dataset, even though the authors warn about it.

EDIT: There is one more issue with Urbansound8k folds, and that is that the difficulty of the various folds is quite different. So one should ideally report the performance across all folds (mean/std or boxplot). But this is a minor issue compared to data leakage.

PS: Nice use of Comet.ml platform this, collaborating online on improving the experimental setup :)

Re: Applying machine learning and deep learning methods to audio analysis

#15

Post by alon7 »

m0zg wrote: post #3

As one of the long-suffering Comet.ml customers, I wish they'd spend more time working on their site's performance and less on writing blog posts. It takes multiple seconds for graphs to render, and leaving any part of Comet.ml UI open in the browser leads to spinning fans and quick battery drain when working from a laptop. The logging component will sometimes hang without a warning and hang your training session as…

We're actually very happy with Comet and have been using it on v large projects (>50 researchers, 10k models). You can reduce the refresh interval and the amount of data points reported if things feel slow

Re: Applying machine learning and deep learning methods to audio analysis

#16

Post by abrichr »

m0zg wrote: post #3

As one of the long-suffering Comet.ml customers, I wish they'd spend more time working on their site's performance and less on writing blog posts. It takes multiple seconds for graphs to render, and leaving any part of Comet.ml UI open in the browser leads to spinning fans and quick battery drain when working from a laptop. The logging component will sometimes hang without a warning and hang your training session as…

I'd love to learn more about your use case. What kind of models are you training? What are you using Comet.ml for?

Thanks!

Re: Applying machine learning and deep learning methods to audio analysis

#17

Post by m0zg »

alon7 wrote: post #15
m0zg wrote: post #3

As one of the long-suffering Comet.ml customers, I wish they'd spend more time working on their site's performance and less on writing blog posts. It takes multiple seconds for graphs to render, and leaving any part of Comet.ml UI open in the browser leads to spinning fans and quick battery drain when working from a laptop. The logging component will sometimes hang without a warning and hang your training session as…

We're actually very happy with Comet and have been using it on v large projects (>50 researchers, 10k models). You can reduce the refresh interval and the amount of data points reported if things feel slow

I don't log that many points as it is: about 4K data points per run in total (windowed average loss and LR every 25-30 batches, eval metrics every epoch), for all metrics combined. I also log the same data to TensorBoard, which renders everything pretty much instantaneously with no issues at all, even though I tell it to not downsample beyond 5K samples per graph.

Re: Applying machine learning and deep learning methods to audio analysis

#18

Post by gidim »

m0zg wrote: post #17
alon7 wrote: post #15

Earlier quoted context omitted.

We're actually very happy with Comet and have been using it on v large projects (>50 researchers, 10k models). You can reduce the refresh interval and the amount of data points reported if things feel slow

I don't log that many points as it is: about 4K data points per run in total (windowed average loss and LR every 25-30 batches, eval metrics every epoch), for all metrics combined. I also log the same data to TensorBoard, which renders everything pretty much instantaneously with no issues at all, even though I tell it to not downsample beyond 5K samples per graph.

M0zg do you mind sending me an email with your project? Happy to look into it. gideon a t comet.ml

Re: Applying machine learning and deep learning methods to audio analysis

#19

Post by gidim »

gidim wrote: post #18
m0zg wrote: post #17

Earlier quoted context omitted.

I don't log that many points as it is: about 4K data points per run in total (windowed average loss and LR every 25-30 batches, eval metrics every epoch), for all metrics combined. I also log the same data to TensorBoard, which renders everything pretty much instantaneously with no issues at all, even though I tell it to not downsample beyond 5K samples per graph.

M0zg do you mind sending me an email with your project? Happy to look into it. gideon a t comet.ml

Also keep in mind that unlike tensorboard we keep your full data series available in the API and only downsample the charts to 15k points.

Re: Applying machine learning and deep learning methods to audio analysis

#20

Post by nikolaskaris »

jononor wrote: post #14
gidim wrote: post #13

Earlier quoted context omitted.

just to clarify - are you referring to this experiment? https://www.comet.ml/demo/urbansound8k/be09e32700cd435fb6b55...

Sure, that demonstrates the issue. Problem is with using train_test_split(X, yy, test_size=0.2..) - this assumes independent samples, which is violated for this dataset (because some come from same source audio files). The easiest (and completely acceptable) is to use one fold as the validation data, one fold for the test set, and the remaining folds as training. This problem is unfortunately quite common even in aca…

Hey jononor — we've updated the post to split the training and test sets based on the folds. Good catch and thanks again for reporting this. Some of the experiments in the project will still have the old code, but the blog post will reflect this new train/test split.
Post reply on HN