Writing and coding by voice with Talon

Talon is looking extremely promising, with its zoom mouse that works via gaze-tracking. Generally coding by voice is a graveyard of abandoned projects. I wrote an overview of the field last year, if anyone's interested in alternatives:

https://medium.com/bambuu/state-of-voice-coding-2017-3d2ff41...

Re: Writing and coding by voice with Talon

#5

Post by melling » Mon, Dec 31, 2018, 10:12 AM UTC

I have a collection of Programming by Voice links here:

https://github.com/melling/ErgonomicNotes/blob/master/progra...

Re: Writing and coding by voice with Talon

#6

Post by malloryerik » Mon, Dec 31, 2018, 10:51 AM UTC

It'd be great to have preconfigured rule sets for various programming languages.

Re: Writing and coding by voice with Talon

#7

Post by shakna » Mon, Dec 31, 2018, 12:21 PM UTC

inciampati wrote: post #2

Is there a similar system available for Linux?

The state of voice on Linux is awful.

I've got something close to the Talon control, without eye tracking, I'm working on now and then, using the Google API for voice interpret, because CMUSphinx was awful (50% accuracy per word, whereas Google was closer to 90%).

I'm hoping that Mozilla Voice when it comes will finally solve this or make it easy to build a decent control system.

Re: Writing and coding by voice with Talon

#8

Post by yorwba » Mon, Dec 31, 2018, 1:16 PM UTC

shakna wrote: post #7

inciampati wrote: post #2

Is there a similar system available for Linux?

The state of voice on Linux is awful. I've got something close to the Talon control, without eye tracking, I'm working on now and then, using the Google API for voice interpret, because CMUSphinx was awful (50% accuracy per word, whereas Google was closer to 90%). I'm hoping that Mozilla Voice when it comes will finally solve this or make it easy to build a decent control system.

Mozilla DeepSpeech has had a release [1] that comes with a pre-trained model achieving 11% WER on clean audio in the LibriSpeech test corpus. That's close to the WER you're getting with Google, but I guess your audio quality isn't as good, so DeepSpeech would perform worse.

Mozilla Common Voice [2] is the project to collect more training data so that DeepSpeech (and other projects) can achieve the accuracy that is known to be possible with the same architecture trained on larger private datasets.

Then there's Facebook's newly released wav2letter++ [3], which claims to achieve better accuracy with the same training data. However, some people have been unable to exactly reproduce those results, getting "only" 5.15% WER [4]. Still better than what Mozilla DeepSpeech can deliver, though.

[1] https://github.com/mozilla/DeepSpeech/releases/latest

[2] https://voice.mozilla.org

[3] https://github.com/facebookresearch/wav2letter

[4] https://github.com/facebookresearch/wav2letter/issues/88

Re: Writing and coding by voice with Talon

#9

Post by eggie » Mon, Dec 31, 2018, 1:34 PM UTC

shakna wrote: post #7

inciampati wrote: post #2

Is there a similar system available for Linux?

The state of voice on Linux is awful. I've got something close to the Talon control, without eye tracking, I'm working on now and then, using the Google API for voice interpret, because CMUSphinx was awful (50% accuracy per word, whereas Google was closer to 90%). I'm hoping that Mozilla Voice when it comes will finally solve this or make it easy to build a decent control system.

Does this mean that Talon is based on private APIs provided in OSX?

Re: Writing and coding by voice with Talon

#10

Post by CodeXs » Mon, Dec 31, 2018, 2:32 PM UTC

Talon is promising but is not open source at its core nor does Nuance has drop support for Dragon Naturally Speaking on OSX.

Caster project seems up-to-date and Dragonfly as well with a new fork that's actively integrating other speech recognition engines.

Combined with Aenea I can dictate code on Linux