Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speech Recognition Is Not Solved

awni.github.io

Post reply on HN
Print view View original

171–180 of 208 posts

Re: Speech Recognition Is Not Solved

#171

Post by flukus »

adrianbg wrote: post #108
johnramsden wrote: post #6

What I would like to see you would be an open source speech recognition system that is easy to use, and works. There is really no match for the proprietary solutions in the open source world right now, and that is disappointing. Not to say that there aren't open source speech recognition systems, they just aren't completely usable in the way proprietary solutions are. A lot of research goes into open source speech re…

You are correct that the problem is data. Kaldi is hard to use, but making it easier to use isn't as hard as getting good training data. Mozilla's project is a good start for some purposes. One flaw with it is that they're having people read sentences. When people read, they tend to speak more clearly than when they're figuring out what to say on the fly. This means models trained with Mozilla's data will tend to nee…

It's only a data issue if they're trying to reproduce google voice. Personally I got much better results from dragon naturally speaking 20 years ago than I get from google voice today, the cost was that you had to train it yourself first, but the benefit was it was trained for you, not "everyone". The later is the approach I'd prefer to see mozilla/OSS take.

Re: Speech Recognition Is Not Solved

#172

Post by jrkdgdjenr »

Cacti wrote: post #131
dragontamer wrote: post #125

Earlier quoted context omitted.

> The TPU architecture isn't that weird Its "weird" in the ways that matter: there's no commodity hardware in existence that replicates what a TPU does. The only place to get TPUs is through Google's cloud services. CPUs are basically Von Neumann Architecture. GPUs (NVidia and AMD) are basically SIMD / SIMT systems. Google's TPU is just something dramatically different, optimized yes for Matrix Multiplication, but it…

> but its not something you can buy and use offline. But you will. The entire point is to put this in a phone, so you can distribute a trained neural net in a way that people can actually use without a desktop and $500-$4,000 GPU.

If this were true i’d expect to see some effort to open and standardize the hardware. Otherwise what’s the point?

Re: Speech Recognition Is Not Solved

#173

Post by tmulc18 »

In the caption for figure 2 there is this quote, "Notice the humans are worse at transcribing the non-American accents." But when I look at the bar graph, I see the word error rates of the model is much higher than the word error rates of humans for most of the non-American accents. Am I missing something?

Re: Speech Recognition Is Not Solved

#174

Post by adrianbg »

woodson wrote: post #155
adrianbg wrote: post #133

Earlier quoted context omitted.

The 8.5% in this file is what you'd compare to Microsoft and IBM's recent ~5% results. https://github.com/kaldi-asr/kaldi/blob/master/egs/fisher_sw... Kaldi hasn't been in first place on that dataset recently, but it was a few years ago. On other more researchy datasets (eg. for distant speakers or languages other than English), the best system is often based on Kaldi.

One reason for the discrepancy between quoted numbers is that, if you are only after pushing that number down and not particularly interested in getting a scalable system, then you are free to run as many systems as you like in as many configurations as possible and then try to combine their outputs (ROVER etc.).

Yeah. I don't get the impression that the Kaldi core team has been trying very hard recently to get SOTA on eval2k/switchboard. This number uses one acoustic model with a trigram LM decode + fourgram rescoring -- there isn't even a neural net language model in there. If I remember correctly, Microsoft's first "human parity" result used something like three acoustic models and at least four types of language models. This Kaldi model is competitive with the best single acoustic model Microsoft used.

Re: Speech Recognition Is Not Solved

#175

Post by woodson »

adrianbg wrote: post #174
woodson wrote: post #155

Earlier quoted context omitted.

One reason for the discrepancy between quoted numbers is that, if you are only after pushing that number down and not particularly interested in getting a scalable system, then you are free to run as many systems as you like in as many configurations as possible and then try to combine their outputs (ROVER etc.).

Yeah. I don't get the impression that the Kaldi core team has been trying very hard recently to get SOTA on eval2k/switchboard. This number uses one acoustic model with a trigram LM decode + fourgram rescoring -- there isn't even a neural net language model in there. If I remember correctly, Microsoft's first "human parity" result used something like three acoustic models and at least four types of language models. T…

Fully agree. I think their work on training data augmentation (e.g., their ICASSP paper, http://danielpovey.com/files/2017_icassp_reverberation.pdf, or the ASPiRE model before) has a bigger impact on the practical usefulness of ASR than getting an X% relative improvement over the previous SOTA on the eval2000 set.

Re: Speech Recognition Is Not Solved

#176

Post by adrianbg »

flukus wrote: post #171
adrianbg wrote: post #108

Earlier quoted context omitted.

You are correct that the problem is data. Kaldi is hard to use, but making it easier to use isn't as hard as getting good training data. Mozilla's project is a good start for some purposes. One flaw with it is that they're having people read sentences. When people read, they tend to speak more clearly than when they're figuring out what to say on the fly. This means models trained with Mozilla's data will tend to nee…

It's only a data issue if they're trying to reproduce google voice. Personally I got much better results from dragon naturally speaking 20 years ago than I get from google voice today, the cost was that you had to train it yourself first, but the benefit was it was trained for you, not "everyone". The later is the approach I'd prefer to see mozilla/OSS take.

That's fair. I think both approaches are useful in different situations. Mozilla seems to be focussing on Siri-like use-cases as opposed to dictation. Even for dictation, for many people having to train the system themselves is more work than they're willing to do. I'm sure what you want will exist eventually :)

Re: Speech Recognition Is Not Solved

#177

Post by adrianbg »

woodson wrote: post #149
adrianbg wrote: post #110

Earlier quoted context omitted.

Although some models are end-to-end neural nets, most of the ones in production (and all of the ones that get state of the art results) only use a neural net for one part of the process. Lots of people are as surprised as you, but that's the way it is. Edit: I should say that in state of the art results there tend to be multiple components, including multiple neural nets and the tricky "decode graph" that gok and I a…

Separating acoustic model and decoding graph search makes sense since you would need a huge amount of (correctly!) transcribed speech for training. See, for example, this paper by Google [1], where they used 125,000 hours (after filtering out the badly transcribed ones from the original 500,000 hours of transcribed speech) for training an end-to-end acoustic-to-word model. Good "old-school" DNN acoustic models can al…

Yes, exactly. I do wonder whether a similarly good end-to-end system could be trained by constraining the alignments as I've seen done in some papers.

Re: Speech Recognition Is Not Solved

#178

Post by dboreham »

gsnedders wrote: post #99
dboreham wrote: post #92

Earlier quoted context omitted.

Parliamo Glasgow: https://www.youtube.com/watch?v=TfCk_yNuTGk I also grew up in Fife, although my parents paid good money so I would have an Edinburgh accent. Glasgow was like a foreign country to us...

> I also grew up in Fife, although my parents paid good money so I would have an Edinburgh accent. I grew up in St Andrews, both of my parents having grown up in England, and went through speech therapy as a young child (due to dyspraxia); unsurprisingly, with that, you can imagine my accent is much closer to RP than any broad Fife accent, though most of my speech is definitely Standard Scottish English.

Well...St Andrews really isn't Fife ;)

Re: Speech Recognition Is Not Solved

#179

Post by adrianbg »

twic wrote: post #154
adrianbg wrote: post #102

Earlier quoted context omitted.

If you think that's bad, check out the Geordie accent from Newcastle: https://www.youtube.com/watch?v=ZY4TT3VtR8o According to Wikipedia it's "a direct continuation and development of the language spoken by the Anglo-Saxon settlers" of the region. https://en.wikipedia.org/wiki/Geordie

Jimmy Nail! Nevertheless, Rab C. Nesbitt remains my high water mark for impenetrability in British television: https://www.youtube.com/watch?v=BkC0jCoedHc

I defer to your obvious familiarity with British TV.

Re: Speech Recognition Is Not Solved

#180

Post by hprotagonist »

hzhou321 wrote: post #148

I suspect we (human) are very bad at hearing, computers already are doing are much better jobs than us. However, I suspect for most our communication, we do not actually hear the conversation -- we guess the conversation. Only when evidence such as facial response or out-of-context words is caught then we actually try to hear. Even then, we are still trying to second guess. Computers cannot compete with our guessing…

we are exceptionally good at hearing. Our ears are truly exquisitely precise things.Where "hearing" ends and "understanding" begins is, shall we say, a point of contention.

My vote is for the auditory nerve as the inflection point. Anything above the cochlear nucleus on the auditory pathway is black voodoo magic and anyone who claims they understand it needs to reevaluate their kool-aid intake.

Post reply on HN