Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

github.com

Post reply on HN
Print view View original

31–40 of 43 posts

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#31

Post by snadal »

bmc7505 wrote: post #28
kcorbitt wrote: post #26

This is really exciting. I previously worked at a startup for that could have benefited enormously from even 90% accurate speech recognition. As of six months ago when I last looked, there were no open source speech-to-text libraries with anything approaching the performance of the proprietary work by Google, Microsoft, Baidu, etc. The closest thing was CMU Sphinx, but its accuracy was unacceptable. Props to the auth…

The CMU Sphinx project as it stands is basically dead. Even though they recently implemented some sequence-to-sequence deep learning techniques for g2p [1], the core stack is still based on an ancient GMM/HMM pipeline, and current state of the art projects (even open source ones) have leapfrogged it in terms of accuracy. If you're implementing offline speech recognition today, start with something like this or Kaldi-…

We've worked in the past with CMU Sphinx too, and it is absolutely amazing the advances in this area in the last months.

A little bit off-topic, but do you know any recent work or paper for speech recognition in language teaching area ? (I mean, analysing and rating accuracy of speaker, detect incorrect pronunciation of phones, and so on)

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#32

Post by brandoncarl »

To the authors: did you any of your own recordings? I've used my own and clips online, in WAV and other formats, at various sampling rates.

All of the results come back gibberish. The results in the training data seem just fine. Curious if you've tested the above to ensure it didn't overfit.

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#33

Post by bmc7505 »

snadal wrote: post #31
bmc7505 wrote: post #28

Earlier quoted context omitted.

The CMU Sphinx project as it stands is basically dead. Even though they recently implemented some sequence-to-sequence deep learning techniques for g2p [1], the core stack is still based on an ancient GMM/HMM pipeline, and current state of the art projects (even open source ones) have leapfrogged it in terms of accuracy. If you're implementing offline speech recognition today, start with something like this or Kaldi-…

We've worked in the past with CMU Sphinx too, and it is absolutely amazing the advances in this area in the last months. A little bit off-topic, but do you know any recent work or paper for speech recognition in language teaching area ? (I mean, analysing and rating accuracy of speaker, detect incorrect pronunciation of phones, and so on)

> Do you know any recent work or paper for speech recognition in language teaching area?

What you're describing is called "speech verification". Language education is an application I'm personally very interested in, and one that almost no one discusses in the speech community (I assume because of machine translation), so if you find any research papers please let me know! I wrote a little about it: http://breandan.net/2014/02/09/the-end-of-illiteracy/

The task is actually much simpler than STT. You display some text on the screen, wait for an audio sample, then check the model's confidence that the sample matches the text. If the confidence is lower than some threshold, then you play the correct pronunciation through the speaker. The trick is doing this rapidly, so a fast local recognizer is key. I've got a little prototype on Android, and it's pretty neat for learning new words. I'd like to get it working for reading recitation, but that's a lot of work.

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#34

Post by bmc7505 »

echelon wrote: post #12

Did the original WaveNet text to speech demo come with a paper or source code? (I didn't see either.) I'm interested in techniques, particularly neural network-related, to improve the quality of my Donald Trump text to speech engine [1]. Does anyone on HN do active research in this field? Could I pick your brain for a survey of the best papers (especially review papers) on the subject? [1] http://jungle.horse

> Did the original WaveNet text to speech demo come with a paper or source code?

Paper, yes. [1] Source code, no.

[1] https://arxiv.org/pdf/1609.03499.pdf

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#35

Post by kondro »

RandomInteger4 wrote: post #20

How much Bandwidth is consumed from voice communications such as when speaking to someone on Skype or over the phone, vs. the same words transmitted via text? Perhaps future communication applications can have a WaveNet on either end, which learns the voice of the person you're communicating with and then only sends text after a certain point in the conversation? I'm coming at this from a point of ignorance though, s…

Less than 8kbps in most voice. It pales in comparison to the quantity of bandwidth consumed each day on video.

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#36

Post by gwern »

epistasis wrote: post #23
imaginenore wrote: post #19

Earlier quoted context omitted.

What's super-human parity? And how do you achieve it using humans?

Presumably more accurate than a single human, and you can do it with multiple humans and reaching a consensus. I remember an anecdote in physics class where an experiment required counting a certain number of events in time. A single person would occasionally blink and miss an event. But if you had two people, and you count how many people observed each event, you can solve for super-human accuracy using the estimate…

Ensembles are well-known to be more accurate. But this is not an advantage exclusive to humans: NNs ensembled will do better than any of the individual NNs.

There's no reason one couldn't train 5 or 10 RNNs for transcription and ensemble them. (Indeed, one cute trick this ICLR was how to get an ensemble of NNs for free so you don't have to spend 5 or 10x time training: simply lower the learning rate during training until it stops improving, save the model, then jack the learning rate way up for a while and start lowering it until it stops improving, save that model, and when finished, now you have _n_ models you can ensemble.) And computing hardware is cheaper than humans, so it will be cheaper to have 5 or 10 RNNs process an audio file than it would be to have 2 or 3 humans independently check, so the ensembling advantage is actually bigger for the NNs in this scenario.

Humans still have the advantage of more semantic understanding, but RNNs can be trained on much larger corpuses and read all related transcripts, so even there the human advantage is not guaranteed.

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#37

Post by gwern »

londons_explore wrote: post #10

Looking at the training loss graph, it looks like training for more time would produce even better results... Anyone want to volunteer a few weeks of GPU time to train this better?

Training loss pretty much always decreases. NNs are extremely powerful models, so they can overfit most data. What you want to see is the validation loss graph.

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#38

Post by visarga »

gwern wrote: post #36
epistasis wrote: post #23

Earlier quoted context omitted.

Presumably more accurate than a single human, and you can do it with multiple humans and reaching a consensus. I remember an anecdote in physics class where an experiment required counting a certain number of events in time. A single person would occasionally blink and miss an event. But if you had two people, and you count how many people observed each event, you can solve for super-human accuracy using the estimate…

Ensembles are well-known to be more accurate. But this is not an advantage exclusive to humans: NNs ensembled will do better than any of the individual NNs. There's no reason one couldn't train 5 or 10 RNNs for transcription and ensemble them. (Indeed, one cute trick this ICLR was how to get an ensemble of NNs for free so you don't have to spend 5 or 10x time training: simply lower the learning rate during training u…

Yeah, but you don't want to run an ensemble of 10 RNNs on your phone, or in the cloud for that matter, when you got billions of queries. It's too expensive.

In practice the ensemble model is compactly transferred into a single network. In order to do that, they train a new network to copy the outputs of the ensemble, exploiting "dark knowledge".

Recurrent Neural Network Training with Dark Knowledge Transfer - https://arxiv.org/abs/1505.04630v5

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#39

Post by RandomInteger4 »

dest wrote: post #25
RandomInteger4 wrote: post #20

How much Bandwidth is consumed from voice communications such as when speaking to someone on Skype or over the phone, vs. the same words transmitted via text? Perhaps future communication applications can have a WaveNet on either end, which learns the voice of the person you're communicating with and then only sends text after a certain point in the conversation? I'm coming at this from a point of ignorance though, s…

text communication is much lighter (a few bytes/s vs kb/s) but you may miss the non verbal contents of voice

By non-verbal do you mean like ambient sound? Dogs barking, child yelling, garbage truck garbage trucking? I don't know. If they can do voice, then it might be possible to do ambient sounds of there is a separate nets trained with a library of ambient sounds where it's tuned not to be the same every time the sound plays like how when you have tiled graphics, there are algorithms that remove the unnatural sameness from one tile to the next.

This could have interesting implications for Foley-artists of the 21st century.

How likely would such a tech help lower budget companies who want to implement voice communication within their software, say for video games or similar?

Hmm, now this has me wondering what implications this has for voice acting as well.

EDIT: We can call the ambient sound symbols sent over the wire "Soundmojis" or "amojis" or "audiomojis"

Re: Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition

#40

Post by IshKebab »

Can someone explain why MFCC is used rather than allowing the neural network to learn from the raw waveform? I looked back in the literature and the intention of MFCC & PLP seems to be to remove speaker-dependent features from the audio in order to reduce the dimensionality of the input. But I though the whole point of neural nets is that they can learn from very high dimensional inputs no?

I had a go at implementing wave->phoneme recognition using a simple neural net and it seemed to work pretty well.

Post reply on HN