Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Voice Recognition and Text to Speech in Python

ggulati.wordpress.com

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Voice Recognition and Text to Speech in Python

#31

Post by dharma1 »

amelius wrote: post #18
giancarlostoro wrote: post #4

It really would be amazing to be able to get voice recognition software that covers at least recognizing a small enough fraction of our language to be useful without having to reach the cloud. It is definitely a dream I hope we one day achieve, thanks for the article, will test it on my day off and play with it a bit.

> It really would be amazing to be able to get voice recognition software that covers at least recognizing a small enough fraction of our language to be useful without having to reach the cloud. Well, I guess at some point this functionality will become part of the OS. When OSX and Windows offer this, then Linux cannot stay behind, and we will see open source speech recognition libraries.

I'm hoping some open source libraries will come out of this - it's state of the art

https://github.com/baidu-research/warp-ctc

Re: Voice Recognition and Text to Speech in Python

#32

Post by danso »

pbw wrote: post #30
danso wrote: post #20

FWIW, IBM has a wonderful speech to text API...I've put together a repo of examples and Python code: https://github.com/dannguyen/watson-word-watcher One of the great things about it is its word-level time stamp and confidence data that it returns...here's a few super cuts I've made from the presidential primary debates: https://www.youtube.com/watch?v=VbXUUSFat9w&list=PLLrlUAN-Lo... It's not perfect by any means, bu…

It took me a while to understand what you did here. I was waiting for some kind of subtitles showing the recognition ability. But you are saying you performed speech recognition on the full video then edited it according to where the words you targeted were found. I liked the bomb/terrorist one, the others didn't seem to be "saying" anything.

Yeah, I was a bit lazy...I could have used moviepy (which I currently use, but merely as a wrapper around ffmpeg) to add subtitles to show which identified word was identified...I'm hoping to make this into a command-line tool for myself to quickly transcribe things...though making supercuts is just a fun way to demonstrate the concepts.

The important takeaway is that the Watson API parses a stream of spoken audio (other services, such as Microsoft's Oxford, works only on 10-second chunks, i.e. optimized for user commands) and tokenizes it...what you get is a timestamp for when each recognized word appears, as well as a confidence level and alternatives if you so specify. Other speech-transcription options don't always provide this...I don't think PocketSphinx does, for example. Or sending your audio to a mTurk based transcription service.

Here's a little more detail about The Wire transcription, along with the JSON that Watson returns, and a simplified CSV version of it:

https://github.com/dannguyen/watson-word-watcher/tree/master...

Re: Voice Recognition and Text to Speech in Python

#34

Post by th-ai »

danso wrote: post #20

FWIW, IBM has a wonderful speech to text API...I've put together a repo of examples and Python code: https://github.com/dannguyen/watson-word-watcher One of the great things about it is its word-level time stamp and confidence data that it returns...here's a few super cuts I've made from the presidential primary debates: https://www.youtube.com/watch?v=VbXUUSFat9w&list=PLLrlUAN-Lo... It's not perfect by any means, bu…

Youtube speech recognition is getting quite good, at least for talking heads in English. Are there additional top tier API's other than the IBM?

Re: Voice Recognition and Text to Speech in Python

#35

Post by lovelearning »

danso wrote: post #27
lovelearning wrote: post #24

Earlier quoted context omitted.

Pocketsphinx/Sphinx with a small, use-case specific dictionary showed much better accuracy for my accent and speech defects, than any of these cloud based recognition systems. I used a standard acoustic model, but it probably would have been even more accurate had I trained a custom acoustic model. For simple use cases like home automation or desktop automation, I think it's a more practical approach than depending o…

I haven't tried out Pocket Sphinx myself...could you describe the training process, e.g. how long did it take, how much audio did you have to record, how easy was it to iterate to improve accuracy?

PocketSphinx/Sphinx use three models - an acoustic model, a language model and a phonetic dictionary. I'm no expert, but as I understand them, the acoustic model converts audio samples into phonemes(?), the language model contains probabilities of sequences of words, and the phonetic dictionary is a mapping of words to phonemes.

Initially, I just used standard en-us acoustic model, US english generic language model, and its associated phonetic dictionary. This was the baseline for judging accuracy. It was ok, but neither fast nor very accurate (likely due to my accent and speech defects). I'd say it was about 70% accurate.

Simply reducing the size of the vocabulary boosts accuracy because there is that much less chance of a mistake. It also improves recognition speed. For each of my use cases (home and desktop automation), I created a plain text file with the relevant command words. Then used their online tool [1] to generate a language model and phonetic dictionary from it.

For the acoustic model, there are two approaches - "adapting" and "training". Training is from scratch, while adapting adapts a standard acoustic model to better match personal accent or dialect or speech defects.

I found training as described [2] rather intimidating, and never tried it out. This is likely to take a lot of time (a couple of days atleast I think, based on my adaptation experience).

Instead I "adapted" the en-us acoustic model [3]. About an hour to come up with some grammatically correct text that included all the command words and phrases I wanted. Then reading it aloud while recording using Audacity. I attempted this multiple times, fiddling around with microphone volume and gain, trying to block ambient noise (I live in a rather noisy env), redoing it, final take. Took around 8 hours altogether with breaks. Finally generating the adapted acoustic model. About an hour.

About 95% of the time it understands what I say. About 5% of the time, I have to repeat. Especially with phrases.

Did this on both a desktop and raspberry pi. The Pi is the one managing home automation. I'm happy with it :)

[1]: http://www.speech.cs.cmu.edu/tools/lmtool-new.html

[2]: http://cmusphinx.sourceforge.net/wiki/tutorialam

[3]: http://cmusphinx.sourceforge.net/wiki/tutorialadapt

PS: Reading their documentation and searching for downloads takes more time than the actual task. They really need to improve those.

Re: Voice Recognition and Text to Speech in Python

#36

Post by danso »

th-ai wrote: post #34
danso wrote: post #20

FWIW, IBM has a wonderful speech to text API...I've put together a repo of examples and Python code: https://github.com/dannguyen/watson-word-watcher One of the great things about it is its word-level time stamp and confidence data that it returns...here's a few super cuts I've made from the presidential primary debates: https://www.youtube.com/watch?v=VbXUUSFat9w&list=PLLrlUAN-Lo... It's not perfect by any means, bu…

Youtube speech recognition is getting quite good, at least for talking heads in English. Are there additional top tier API's other than the IBM?

AT&T has its own "Watson"...but it requires signing up for a premium account, which I think involves an upfront cost:

http://developer.att.com/apis/speech

Twilio has one that also requires payment:

https://www.twilio.com/docs/api/rest/transcription

It limits input audio to 2 minutes. And I would have to guess that its model is specifically tuned to phone messages, i.e. one speaker, relatively clear and focused audio, and certain probabilities of phrases.

Re: Voice Recognition and Text to Speech in Python

#37

Post by squeaky-clean »

Very cool! I just started playing with speech recognition in Python for home automation this week. I'm controlling some WeMo switches and my PC with an Android Tablet using Autovoice, and it works well as a proof-of-concept, but Autovoice doesn't always register commands, and the "Okay, Google" speech to text can be slow sometimes. I'd like it to take less than 5 seconds between saying "TV Off" and the TV actually turning off., with Autovoice it's anywhere from 3s to 25s depending on the lag. I also figure with real code, I can get commands that are more flexible than Autovoice's regex.

Aside from circumventing lag, I can also give it some personality. I want to name it Marvin, after the robot from H2G2, so that I can say:

"Marvin, turn the TV off"

"Here I am, brain the size of a planet, and you ask me to turn off the tv. Call that job satisfaction, 'cause I don't."

Re: Voice Recognition and Text to Speech in Python

#38

Post by bainsfather »

skykooler wrote: post #28
amelius wrote: post #13

Earlier quoted context omitted.

> It really would be amazing to be able to get voice recognition software that covers at least recognizing a small enough fraction of our language to be useful without having to reach the cloud. Are there any academic groups working on this topic, and do they have prototype implementations?

Julius [1] is a pretty good offline speech recognition engine. In my tests it seems to have about 95% accuracy in grammar-based models, and it supports continuous dictation. There is also a decent Python module which supports Python 2, and Python 3 with a few tweaks. HOWEVER: The only continuous dictation models available for Julius are Japanese, as it is a Japanese project. This is mainly an issue of training data.…

After a quick look, it seems Julius doesn't use the new deep-learning stuff?

In terms of data, http://www.openslr.org/12/ says it has 300 hours + of speech+text from librivox audiobooks. Using Librovox recordings seemed a great idea for making a freely available large dataset.

Re: Voice Recognition and Text to Speech in Python

#40

Post by super_mario »

giancarlostoro wrote: post #4

It really would be amazing to be able to get voice recognition software that covers at least recognizing a small enough fraction of our language to be useful without having to reach the cloud. It is definitely a dream I hope we one day achieve, thanks for the article, will test it on my day off and play with it a bit.

Every Mac with OS 10.9 and later comes with speech recognition software that works without internet access and is really really good. You can dictate entire documents, emails, even have it type commands in terminal etc. In OS X 10.11 you can even drive the UI by only speaking.
Post reply on HN