Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing and coding by voice with Talon

blakewatson.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Writing and coding by voice with Talon

#21

Post by lunixbochs »

melling wrote: post #20
lunixbochs wrote: post #13

Earlier quoted context omitted.

Hi, I’m the sole developer behind Talon. It’s not open source for several reasons, one of which is that I work on it full time, live on my savings, and give it away for free (because the best thing I can get out of releasing Talon is reducing hand pain in others at scale, not money). It has eye/head tracking, noise recognition, and an extremely advanced scripting engine, so it’s not just a speech recognition project.…

How good is Talon’s speech recognition? It would be nice to have a better native Mac solution.

Well, even when using it with Dragon, Talon only uses the speech recognition part. So you get Dragon’s recognition accuracy but don’t need to worry about most of the bugs.

The builtin engine is a much nicer experience than Dragon (very fast startup, no config), but simply isn’t as good at recognizing English, and there’s also a weird behavior in it I haven’t worked around yet that makes it harder to mix English with commands (it greedily prioritizes commands sometimes). The accuracy is quite good for American accents at least. I found it to be consistently more accurate than wav2letter (haven’t tried ++) and DeepSpeech with some simple test audio.

Re: Writing and coding by voice with Talon

#22

Post by lunixbochs »

swlkr wrote: post #18

I noticed the keyword for dash-separated-words is spinal, I always thought it was called kebab case. I guess it's different for different people

I think spinal was the command used by voicecode, and the community repo was put together by ex-voicecode users. You can change a command easily by editing the script that defines it anyway (changes take effect immediately without a restart).

Re: Writing and coding by voice with Talon

#23

Post by lunixbochs »

porpoisely wrote: post #19
rijoja wrote: post #12

Any actual users around to testify about this?

I was wondering about testimonials myself. It sounds wonderful for people with physical impairments who can't use a keyboard or mouse. But in terms of productivity or ease of use for most developers, are there any benefits to using voice over a keyboard and mouse?

This is an important question for me to be able to answer, because if I can’t convince you it’s worth splitting your time between alt input (like voice and eye tracking) and keyboard/mouse, I can’t reduce your chance of developing RSI.

There’s the fear approach (I personally believe >50% of people who use computers full-time will develop at least minor hand injuries), but I don’t think fear is enough on its own.

The most promising thing I think is workflow improvements. For voice, the ability to issue commands like “next song” while typing feels amazing. You can also be more specific about many commands, like “focus chrome” is nicer than mashing cmd-tab a bunch or binding a key to each app. For eye tracking, I think autoscrolling text as you read and jumping your mouse to the right spot when you look at your second monitor are two big ones.

Re: Writing and coding by voice with Talon

#25

Post by dwiel »

lunixbochs wrote: post #23
porpoisely wrote: post #19

Earlier quoted context omitted.

I was wondering about testimonials myself. It sounds wonderful for people with physical impairments who can't use a keyboard or mouse. But in terms of productivity or ease of use for most developers, are there any benefits to using voice over a keyboard and mouse?

This is an important question for me to be able to answer, because if I can’t convince you it’s worth splitting your time between alt input (like voice and eye tracking) and keyboard/mouse, I can’t reduce your chance of developing RSI. There’s the fear approach (I personally believe >50% of people who use computers full-time will develop at least minor hand injuries), but I don’t think fear is enough on its own. The…

Similarly to "next song", I enjoy using it for things I dont use often enough to justify memorizing a shortcut or short phrase. "Connect to vpn", "move window desk 5", "rename variable", "activate virtual environment", etc. Similar to creating aliases in the terminal, but for more than just the terminal and with easy to remember spoken phrases instead of terse abbreviations: `lsa -> ls -al`.

That said, once you are proficient, writing code can definitely be as fast or faster than skilled keyboard.

Re: Writing and coding by voice with Talon

#26

Post by lunixbochs »

j88439h84 wrote: post #24

Hi linuxbochs, I'd love to contribute to a project like this, is there a plan to open source it in the future? Thanks!

The “core” is not open source, which is mostly low-level platform integration code. Talon is basically a pile of APIs built around a Python scripting engine. My goal is to make all the user facing features either completely defined in open source user scripts/plugins or fully scriptable/configurable. You can already contribute to the user scripts. Someone could even have a project like Caster that supports using the Talon APIs but is itself fully open source.

I have open-sourced over 100 of my projects. My current decision is to not open-source Talon at least as long as it is my full-time job for no real salary. I’m putting a lot of work into it and giving it away for free, this is what you get.

Re: Writing and coding by voice with Talon

#28

Post by nash8350 »

geewee wrote: post #4

Talon is looking extremely promising, with its zoom mouse that works via gaze-tracking. Generally coding by voice is a graveyard of abandoned projects. I wrote an overview of the field last year, if anyone's interested in alternatives: https://medium.com/bambuu/state-of-voice-coding-2017-3d2ff41...

Check out Precision Gaze Mouse https://precisiongazemouse.com/. It allows you to move the mouse hands-free, which can save time versus moving your arm back and forth. It's also very accurate because it tracks both eye and head movement.

Re: Writing and coding by voice with Talon

#29

Post by nash8350 »

CodeXs wrote: post #10

Talon is promising but is not open source at its core nor does Nuance has drop support for Dragon Naturally Speaking on OSX. Caster project seems up-to-date and Dragonfly as well with a new fork that's actively integrating other speech recognition engines. Combined with Aenea I can dictate code on Linux

I use Caster every day. In fact I'm writing this using Dragon & Caster. There are open source scripts for most programs like Atom and Visual Studio code. Customizing them is really easy since it's open source and the scripts are just Python.
Post reply on HN