Writing and coding by voice with Talon

I use talon (and previously voicecode) for nearly all interactions with my computer for the past few years. Talon+dragon is definitely the state of the art and is quickly improving. Controlling the computer by voice has completely changed my understanding of user interfaces.

One difference between voice vs keyboard/mouse is that the number of 'commands' that can be in scope at any moment is much much larger. With a mouse you must sacrafice screen space for each command, for keyboard shortcuts the user must memorize often somewhat arbitrary key combinations for every new app/command.

With voice, there are still descoverability and some memorization issues, but they feel a lot different. I can remember the command "rename variable" much more easily than "command-alt-r", or was it "command-shift-r"? These commands can also be common across editors, reducing the friction of trying different editors.

One other observation is how I now think of apps more as (often poorly defined) sets of APIs whose inputs are keyboard/mouse events. I really wish more apps had a cleaner method of exposing this API. Some apps have a command pallet or something similar where a keyboard shortcut brings up a auto completed list of all possible actions available, which is still clumsy, but workable. Atom and jupyter notebooks both have this. Something more direct would be very nice.

Hi, I’m the sole developer behind Talon. It’s not open source for several reasons, one of which is that I work on it full time, live on my savings, and give it away for free (because the best thing I can get out of releasing Talon is reducing hand pain in others at scale, not money). It has eye/head tracking, noise recognition, and an extremely advanced scripting engine, so it’s not just a speech recognition project. It also might come to Linux sooner than you think.

It doesn’t depend on Dragon at all, and Nuance dropping support for Dragon 6 isn’t a huge problem yet as long as you can still buy it (Talon has a builtin speech engine, and also fixes/works around most of the long-standing problems in Dragon Mac, and I’m probably more able to help with problems than their support). Integrating more engines is on the roadmap, but slightly after multi platform support (since there’s already a good free engine on Mac).

Let me know if you have any questions or if there’s anything I can do to make Talon work better for your use case.

It’s also worth reading back through my HN comments as I’ve talked in detail about Talon there.

It’s not locked to macOS at all. I plan to support win/lin/mac equally, Talon has its own grammar compiler and engine-independent word parser. The main hold up for porting is all of the interaction APIs like key simulation, drawing overlays on the screen, that sort of thing.

The best I know of for Linux is Aenea, which requires you to run Dragon in a virtual machine or on another computer. I will have Talon ported to Linux at some point, and that will include coming up with speech engine options for Linux. I think one of the better options is to make Windows Dragon work well in WINE. (Despite the numbers being impressive on DeepSpeech and wav2letter++, they’re not yet optimized for continuous recognition, which means you need to feed them chunks of finished recordings, and their letter based recognition requires weird fuzzy matching if you want command grammars that I have yet to solve. I’ve actually considered using the fastest continuous engine I can find, then feeding the sound into a better open source non-continuous engine after each recognition)

I was wondering about testimonials myself. It sounds wonderful for people with physical impairments who can't use a keyboard or mouse. But in terms of productivity or ease of use for most developers, are there any benefits to using voice over a keyboard and mouse?