DeepSpeech 0.6 • Hacker News Board

Re: DeepSpeech 0.6

#61

Post by jcims » Thu, Dec 05, 2019, 5:27 PM UTC

yorwba wrote: post #60

jcims wrote: post #53

Earlier quoted context omitted.

I totally understand where you're coming from, but the reality is that there is a lot of imbalance and delay in this type of assessment and educational support already. My wife has been an English language arts teacher and instructional coach for almost 20 years. While they definitely have standard diagnostics and regular assessments to try to keep kids on track, it's essentially best effort. And when you move out of…

> even basic fluency monitoring without any active remediation attempts would be a good addition. How do you define fluency? I guess it involves some combination of speed and accuracy. Speed shouldn't be too hard to measure, but for accuracy you'll probably need a model that's more accurate than 7.5% WER and can handle the difference in vocal range between children and adults. Otherwise the speech disfluencies you wa…

In the US, fluency metrics in ELA vary across standards/states but are generally coarse grained and somewhat subjective. Here's an example of the standard that a student is expected to meet for fluency in the fifth grade (bottom half): http://www.corestandards.org/ELA-Literacy/RF/5/

You definitely would have to use a model trained from content in the target audience (probably down to the grade level as things change so dramatically from year to year) as well as probably some labeled examples of students with various reading/speech deficiencies. This of course would lead to a lot of challenges from a privacy/regulatory/etc perspective and the entire thing would be challenging from an optics perspective (AI teachers taking over, etc etc).

Edit: One thing to keep in mind with accuracy is that there is no ambiguity what the word should be, the question is how closely the utterance matches the expected sounds of the word.

Re: DeepSpeech 0.6

#62

Post by maxbrunsfeld » Thu, Dec 05, 2019, 5:48 PM UTC

> DeepSpeech v0.6 with TensorFlow Lite runs faster than real time on a single core of a Raspberry Pi 4

This is great news.

I'm not very familiar with the deep learning framework ecosystem. Does anyone know what the simplest way would be to incorporate a DeepSpeech model into a WebAssembly project?

For instance, is it straightforward to compile Tensorflow Lite with emscripten? If not, can this TensorFlow model be run with tensorflow.js? Or can it be converted to some other format that is easier to use with WASM?

Re: DeepSpeech 0.6

#63

Post by reubenmorais » Thu, Dec 05, 2019, 8:41 PM UTC

dabinat wrote: post #58

franciscop wrote: post #15

Tensorflow lite, interesting. Has anyone tried this with Google Coral USB accelerator? I got one laying around but very little experience with ML. I could get the USB Accelerator working with the pretrained posenet after much messing around, but my chances with DeepSpeech are small to none. This seems like the perfect fit though.

To the best of my knowledge, DeepSpeech currently isn’t compatible with any inference or training accelerators (other than NVIDIA GPUs). But the whole point of the TFLite model is that it has low resource requirements so an accelerator for inference shouldn’t be necessary.

TFLite on Android with the NNAPI delegate is theoretically supposed to leverage hardware accelerators. We haven't tested it extensively though.

Re: DeepSpeech 0.6

#64

Post by z3t4 » Thu, Dec 05, 2019, 10:37 PM UTC

I was very exited about this, then I tried it with the pre-trained model and I recorded "Hello this is a test message" 3 times and it was inferred as "a sassanian", "he is a paris" "he states that" ...

Re: DeepSpeech 0.6

#65

Post by whyleyc » Thu, Dec 05, 2019, 11:53 PM UTC

mostlyjason wrote: post #29

reubenmorais wrote: post #24

Earlier quoted context omitted.

Hi! > I guess you prefer an "end-to-end" model over a hybrid HMM/NN model, for simplicity, right? Simplicity and ease of targeting other languages, yes. We're a small team. > As far as I remember, you use CTC, right? I always wondered why you have chosen CTC, and not some better model, like RNN-T, RNA, or some of the streaming attention variants. We started DeepSpeech in 2016, before these recent developments for end…

I really hope you adopt the latest models particularly streaming attention variants. I think you should validate with users the assumption that latency is more important than WER. IMHO the WER is more important than latency improvements in the millisecond range. The most frustrating thing is having to dictate over and over and the transcription is incorrect each time. Consider that the time to a correct transcription…

Thanks for all the hard work you have put in so far @reubenmorais

+1000 to @mostlyjason's comment - Great latency figures mean nothing if the word error rate is high, since it dents confidence in the output (so why use DeepSpeech?) and (as the parent comment notes) necessitates manual error correction.

I would love to see a future release focus on optimizing WER for these reasons.

Re: DeepSpeech 0.6

#66

Post by mncharity » Fri, Dec 06, 2019, 1:09 PM UTC

reubenmorais wrote: post #52

mncharity wrote: post #43

Earlier quoted context omitted.

> There are also examples Fyi, the other 4 example links there are 404. Also in the root README. The linked README's are missing, or diversely suffixed and mislinked.

Ugh, sorry. I fixed it on master. We'll probably cut a v0.6.1 soon to publish small fixes like these.

Thanks for all your work - it's appreciated.

Re: DeepSpeech 0.6

#67

Post by nmstoker » Sat, Dec 07, 2019, 5:58 AM UTC

jcims wrote: post #36

This reminds me of something I would love to see happen but I don't have the skills to put it all together. I really think there's some potential merit to a reading coach app(lication) that listens to someone read and looks for weaknesses/disorders/etc compared to a trained model. It could provide those diagnostics to an educator, guide the content to focus on those, coach the reader directly, etc. It all seems very…

I've seen this referred to online as "Intelligibility Remediation" (a term I always forget when I'm trying to find the details months later!)

Certainly seems like an area with huge potential application.

Re: DeepSpeech 0.6

#68

Post by detaro » Sat, Dec 07, 2019, 7:30 PM UTC

Erlich_Bachman wrote: post #21

therealmarv wrote: post #16

So who will be developing an open source assistant like Goggle Home / Alexa? :)

If you do a 1 minute google search you will find that there are multple open source voice assistant offerings already, for example mycroft.

Most of which are not fully open, e.g. do not have open speech recognition.