Live data from Hacker News

Hearing Heartbeat in Audio and Video: A Deep Learning Project

samcoope.com

11–14 of 14 posts

Re: Hearing Heartbeat in Audio and Video: A Deep Learning Project

#11
post #9
post #6

Earlier quoted context omitted.

From the video they have on youtube, Arnold has over 200bpm and the kid at the end has a 0bpm. That's a pretty useless territory.

That video was just a mock of our product. If you would like to see the results of the performance of the models, have a look at the slides we made for the project (at the bottom of the post). The best performance we got was about 0.7 r.

0.7 r is a bit optimistic I would say. We only got 0.7 for people we have been able to train on. For completely different people it learned nothing with audio spectrograms. I speculate that convolutions on raw audio would be better.

Estimation from video seemed much better. There we actually got 0.5 r on unseen people, which I find very promising.

Re: Hearing Heartbeat in Audio and Video: A Deep Learning Project

#12
I'm not very up on CNN/RNN models yet, beyond the basics, but I do a fair amount of signals analysis. Is your truth data in git? I'd love to work with it. Second, did you try whitening the [audio] data prior to training your network? That might help with exposing some more salient features of your signal. Pretty neat work.

Re: Hearing Heartbeat in Audio and Video: A Deep Learning Project

#13
post #10

That's nice. It doesn't work well enough to be useful yet, but has promise. If this can be made to work, it will be a feature in video chat apps.

Or maybe security trying to detect if someone is in stressful situations. Or maybe in some sort of augmented reality device. May valid use cases.

Re: Hearing Heartbeat in Audio and Video: A Deep Learning Project

#14

I'm not very up on CNN/RNN models yet, beyond the basics, but I do a fair amount of signals analysis. Is your truth data in git? I'd love to work with it. Second, did you try whitening the [audio] data prior to training your network? That might help with exposing some more salient features of your signal. Pretty neat work.

Thanks for the ideas!

The data we used is not on our repo. It came from a database given to us by our university, and there was confidentiality issues when it came to making it publicly available.

Post reply on HN