On Voice Coding

This post starts out talking about expecting to spend around $1,000.

There are at least two cross-platform projects where the biggest expense is a microphone instead of software.

1. My project, Talon. Windows/Linux/Mac support, and a first party local speech recognition engine that is pretty good and getting better. It’s free, but the engine is in a private beta (which is $15/mo to support development, optional if there’s a financial issue).

2. Serenade. They are VC backed. Currently free, unsure about their longer term plans. They use cloud based recognition.

I actively try to switch things up, depending on if I feel something is starting to get uncomfortable. If my mouse hand starts to give signals, I'll switch to using keyboard more. Regardless I'll switch seating position and hand orientation throughout the day.

So in that respect it's unlike smoking I guess.

Some of the voice coding videos I’ve found are a little contrived, or seem practiced (Even some of mine were just copying code from one window to another or demoing one small feature). In this playlist, I do a lot of code from scratch, by competing in Advent of Code 2018 for the first week:

https://www.youtube.com/playlist?list=PL2wTcyeSmhsZuHqbSlLAx...

As I start each day exactly when the problem is revealed, without any prior knowledge of the task, everything about these videos including debugging broken code is completely real.

I had my second surgery just a few months ago. I can type again, but each time I had to use my left hand for months. Initially, it feels like your brain doesn't function properly anymore (not mentioning the psychological effort you have to make in order to be focused on work when you feel your hands are falling apart). Keyboard speed is directly related to how fast you can move your hands to support your thought flow. I tried Kinesis and even a split vertical keyboard (KeyboardIo) but none avoided the pain and numbness that came with typing. The other problem with thumb-cluster keyboards is that your IDE productivity goes to zero. I was faster with just my left hand on a regular keyboard than with both. I think this would be fixable with a good amount of time remapping shortcuts, etc. Now that my hand works again, I think I should start spending time getting used to my KeyboardIO and at least try to buy some time.

The "voice coding" space is maybe not a mess, but far from great or even acceptable. However, there seem to be more recent efforts to make better tools. I would definitely check https://serenade.ai/ out.

The main problem, I think is that "voice coding" is too much focused on editor typing which they can't do right as, when combined with code syntax, it becomes too complex. Instead, they should focus on higher level actions (which btw, Serenade does) along with a different approach to typing. I think Vim is a good example of where editing should be. IntelliJ refactoring is where voice coding should start. With all the AI buzz, it's unbelievable how bad voice recognition is. I'm not talking about "Siri set an alarm", but instead separating context from tone, not having to say things 2-3 times having good response latency, etc.

Lastly, I wish there was simple voice assistance for code navigation - like go to definition, find usages, etc. This is much simpler to "parse" than code structure. Unfortunately, this is not even tackled by any tool as far as I've seen.