Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Future of DeepSpeech / STT after recent changes at Mozilla

discourse.mozilla.org

Post reply on HN
Print view View original

41–50 of 75 posts

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#41

Post by Cybiote »

mrweasel wrote: post #15
jarym wrote: post #3

Maybe we should try to find a list of exactly what they are focussing on going forward instead of the slow drip of things they’re cutting back on (servo, MDN, DeepSpeech...) It’s a sad sad day when you have an organisation getting hundreds of millions in funding and turning away from what’s its good at. The decline has begun in my eyes, it may not become apparent for a few years yet.

Cutting out DeepSpeech seems sensible to me, it’s out of place in the general portfolio of products. It would be nice if Mozilla could tell us what their focus is going to be, but I doubt that Mozilla management know at this point. At this point I’m somewhat concerned that Firefox will be irrelevant in fives years, and I don’t currently feel that Mozilla is communicating clearly that they still care about Firefox. I…

> Cutting out DeepSpeech seems sensible to me, it’s out of place in the general portfolio of products.

I disagree precisely because of the point you make later: "I’m somewhat concerned that Firefox will be irrelevant in fives years".

Functionality provided by deep learning is going to be an important component of many types of software interactions going forward. The logistics of this will be quite different from what we are used to in open source, with the need to fund and coordinate compute, collect and handle data being a more vital aspect compared to the past.

There are STT software, some mentioned in this thread, that match or are even better than DeepSpeech but none of them are as ergonomic. Accounting for the value of time, this means it will be more cost effective to outsource such capabilities to the cloud. Which comes with trade-offs that are difficult to appreciate in the short term: https://news.ycombinator.com/item?id=24236489

I'd say DeepSpeech fits in the mold of Mozilla as a company providing solutions to complicated software problems that are better at respecting the user and their privacy.

In the old days, the most accurate TTS and STT models were built into the OS. These days, you need to call into the cloud to get the best stuff. In [1], Internet Archive complains about the quality of their OCR software. It's not that OCR is so bad, it's that the best OCR is found on Google's and Microsoft's computers. It's possible to cobble something together using open source solutions like EasyOCR, Tesseract+OpenCV but that will only get you part of the way there. What makes the cloud offerings so good is they have enough resources to devote to pre-processing pipelines and architecture tweaks and settings better able to handle edge cases. Most of the mass resides in edge cases.

From my vantage, the future looks to be one of software as thin layers built atop APIs which call into programs running on the servers of a handful of companies. You might not think this a big deal but these software will be the ones scanning the environment, writing the emails, completing the thoughts and planning the calendars for the majority of humans.

[1] https://blog.archive.org/2020/08/21/can-you-help-us-make-the...

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#42

Post by jrochkind1 »

no_wizard wrote: post #8

I don’t know what is going to save Mozilla, really I don’t. I just wish there was a way to “reach” them and discuss how we the internet community could come to an agreement about what they could do to derive value we would pay for. It’s not for a lack of trying on their part for sure, but it feels like just using their browser isn’t all there is to it any more

Has "the internet community" ever "come to an agreement" on literally anything?

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#43

Post by no_wizard »

echelon wrote: post #40
feanaro wrote: post #39

Earlier quoted context omitted.

I feel bad for doing a "me too" comment, but you've nailed exactly my thoughts on the subject. I feel like Mozilla hasn't really tried something like this. Every time it gets suggested, it quickly gets shot down (by other internet commenters) as "can't be done" and "wouldn't generate nearly enough money". Well... maybe not with that CEO salary.

Mozilla can model itself after Microsoft somewhat. Provide a development stack (they're experts at Web and Rust). Make themselves the go-to shop for developers in that realm. Sell them on an OS and editor with support. Partner with Ubuntu. Hell, I would even reach out to Nadella and see if they'd be willing to work with Mozilla on hedging against Google. Mac is becoming locked down and kind of unpleasant to develop o…

This all sounds to me like capital intensive businesses against entrenched players where even the not so average consumer would likely not do more than pay lip service to it unless there was some secret sauce to this that was more compelling to the options

They neeed a good out of the park product in those markets to make any real headway. Too idealistic.

My only thought on this is that they should pivot to be like algolia , focus on Firefox being a reference implementation browser and seek their expertise to the other vendors, maybe. It’s one of the few verticals I can think of that would work strategically Without them having to pivot into things they have no experience with

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#44

Post by nmstoker »

Bootwizard wrote: post #24
albertzeyer wrote: post #20

Earlier quoted context omitted.

There are a lot of open source projects in this space. DeepSpeech is actually one of the outsiders (they are not represented well in the academic community), and also not quite competitive to other software (at least last time I checked). E.g. some very active projects are: * Kaldi ( https://github.com/kaldi-asr/kaldi/ ) obviously, probably the most famous one, and most mature one. For standard hybrid NN-HMM models a…

Can you run audio files through any of these or do they only support audio from microphones?

At the point of them taking in input to process, audio that comes from a microphone or comes from a file is basically just a series of numbers and is the same. So there's no barrier in terms of feasibility.

Whether they're all set up to do that "off the shelf" is a different matter but it should be fairly straightforward to add this to any that lack it and because they're open-source anyone could do a bit of Googling etc and find suitable code to adapt to do it. I know DeepSpeech definitely can take audio from files directly as input as I've used it that way before, and I strongly expect many (or possibly all) of the others could too.

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#45

Post by posguy »

nshm wrote: post #31
echelon wrote: post #30

I'm not sure about Mozilla's efforts in STT, but they were lagging pretty far in TTS. [1] Google/Baidu, universities, and an assortment of Chinese/Japanese/Korean social media companies (Line, etc.) are posting the most compelling TTS research, models, and code. Mozilla's TTS system [2] is an amalgam of some of these models, but it lags pretty far behind state of the art. Mozilla should focus on getting additional re…

It is explainable given that there was a single developer working on TTS. It is hard to compete with big academic teams/industry players this way. I also believe Mozilla team was restricted by a lack of computing resources. They had just a single 8GPU server or so.

Said 8 GPU server was consistently in use for Mozilla DeepSpeech (now renamed Mozilla STT) in training models. Its impressive how far Mozilla got considering how limited their resources were.

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#46

Post by nmstoker »

nshm wrote: post #22
milofeynman wrote: post #10

I was hoping DeepSpeech would lead to in home cloud-less "Alexas". Just ask me for a subscription on it and productize it please.

There are many cloud-less Alexas already, a good one is rhasspy https://rhasspy.readthedocs.io/en/latest/ , it is not based on deepspeech though.

In the repo/docs, it suggests that DeepSpeech is an option for some languages (English & German). Haven't tried it, but with recent(ish) performance improvements in DS it can run on somewhat less powerful computers than used to be the case.

Other options for similar assistants that can also use DS are Mycroft (https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz...) and DragonFire (https://github.com/DragonComputer/Dragonfire)

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#47

Post by posguy »

nshm wrote: post #19
eruleman wrote: post #9

Does anyone know of other open-source projects in the speech-to-text space? DeepSpeech was one of the most promising projects, especially the latest versions...

Try https://github.com/alphacep/vosk-api . It supports 10 languages, works on Android and RPi and also has big and more accurate server models. Other good ones are https://github.com/daanzu/kaldi-active-grammar and https://talonvoice.com/ There are toolkits for research like https://github.com/kaldi-asr/kaldi , https://github.com/espnet/espnet , wav2letter, Espresso, Nvidia/Nemo, https://github.com/didi/athena . You…

Comparing DeepSpeech v0.7.4 to Vosk using plain spoken English samples from male and female speakers, they seem to be performing the same if I use vosk-model-small-en-us-0.3 and the full size DeepSpeech model.

When I use vosk-model-en-us-daanzu-20200328 the result is perfect on many of these tests, though it does not do punctuation or capitalization outside apostrophes. IIRC there is another project on Github that can add basic formatting though.

I am quite surprised with vosk's performance, it even handles odd words like Puget Sound well! Need to test our more accented audio on it, but this is quite exciting.

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#48

Post by posguy »

Bootwizard wrote: post #24
albertzeyer wrote: post #20

Earlier quoted context omitted.

There are a lot of open source projects in this space. DeepSpeech is actually one of the outsiders (they are not represented well in the academic community), and also not quite competitive to other software (at least last time I checked). E.g. some very active projects are: * Kaldi ( https://github.com/kaldi-asr/kaldi/ ) obviously, probably the most famous one, and most mature one. For standard hybrid NN-HMM models a…

Can you run audio files through any of these or do they only support audio from microphones?

DeepSpeech and Vosk can accept audio files, although each wants them formatted in a slightly different mono WAV format.

See my other comment for a comparison of the two: https://news.ycombinator.com/item?id=24248238

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#49

Post by posguy »

Cybiote wrote: post #41
mrweasel wrote: post #15

Earlier quoted context omitted.

Cutting out DeepSpeech seems sensible to me, it’s out of place in the general portfolio of products. It would be nice if Mozilla could tell us what their focus is going to be, but I doubt that Mozilla management know at this point. At this point I’m somewhat concerned that Firefox will be irrelevant in fives years, and I don’t currently feel that Mozilla is communicating clearly that they still care about Firefox. I…

> Cutting out DeepSpeech seems sensible to me, it’s out of place in the general portfolio of products. I disagree precisely because of the point you make later: "I’m somewhat concerned that Firefox will be irrelevant in fives years" . Functionality provided by deep learning is going to be an important component of many types of software interactions going forward. The logistics of this will be quite different from wh…

Based on the testing I just did with Vosk, Mozilla DeepSpeech, Google Speech to Text and Microsoft Azure, I disagree with your arugment that SaaS has the best quality results.

Mozilla DeepSpeech was definitely trailing the bleeding edge, but Vosk using the vosk-model-en-us-daanzu-20200328 model produces very accurate results even on uncommon words, similar in performance to Google & Microsoft (which has generally better formatting than Google's STT)

Try it yourself:

Google: https://cloud.google.com/speech-to-text/ See "Put Speech-to-Text into action" header

Microsoft: https://azure.microsoft.com/en-us/services/cognitive-service... See "Upload File"

Vosk: https://alphacephei.com/vosk/

Had Mozilla provided 4x to 8x more GPU resources and more staff, then their STT would likely be competitive. Other small STT developers can iterate and test much faster due to having more hardware at their disposal.

Re: Future of DeepSpeech / STT after recent changes at Mozilla

#50

Post by halfmatthalfcat »

jrochkind1 wrote: post #42
no_wizard wrote: post #8

I don’t know what is going to save Mozilla, really I don’t. I just wish there was a way to “reach” them and discuss how we the internet community could come to an agreement about what they could do to derive value we would pay for. It’s not for a lack of trying on their part for sure, but it feels like just using their browser isn’t all there is to it any more

Has "the internet community" ever "come to an agreement" on literally anything?

Net neutrality?
Post reply on HN