Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

51–60 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#51

> The biggest issue I've found so far is voice strain My singing teacher started out as a voice therapist. He is still working with a lot of people with damaged voices that are there for therapeutic reasons as well as professional opera singers that there to level-up their game. He says, the best thing you can do for your voice is to sing. Singing will give you the proper "support" from your lungs and train the vocal…

Interesting! Maybe I'll look into a voice trainer.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#52
post #3

As a developer who also has cubital tunnel syndrome, having keyboards that places commonly use modifier keys [0] on your thumbs, such as Kinesis advantage or Ergodox, helps reducing the fatigue and symptoms immensely. [0] it is called Emacs pinkies for a reason, and yes, my CTS is totally emacs’s fault as well.

ye, modifier keys suck. This is why I prefer snake-case over the other casings because it needs no modifier key.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#53
post #50

I don't know why, but I've often thought about the situation the Josh finds himself in. IE - having to find a different way of building software if I couldn't use a keyboard and mouse due to medical condition. The solution I always imagine is paying someone or having my employer pay someone to strong style pair program with me. Perhaps a student, junior developer or even someone unfamiliar with software development e…

> The solution I always imagine is paying someone or having my employer pay someone to strong style pair program with me

Wow, that's some impressive out-of-the-box thinking!

The OP's situation is something I worry about, as I'm worried my neuropathy will reach a point where I can't use my arms to type/mouse all day. I spent some time researching speech input for coding a year ago, and found a few fledgling solutions, but I'd need to expend a lot of time to get something remotely workable.

Your idea is, if I may say so, genius, and having that as a backup sets my mind greatly at ease, so thanks.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#54
Appreciate all the discussion!

I touch on this in the article, but I should say: I'm an edge-case when it comes to CTS. Most cases resolve spontaneously. Part of it is that my nerve dislocates when I bend my elbow, and this mobility causes a bunch of additional friction / inflammation. This happens naturally to about 13% of the population, and mine is particularly pronounced.

All this to say, if you ever do start to get a burning or tingling in your elbows or wrists, or get numbness in the hands, it'll likely go away on its own, or with conservative treatment (a physical therapist is a great first step!).

And, for those who also fall into my unlucky percentile, hopefully it helps to know that there are tools and a community that exist to let you keep working without needing to use your hands at all!

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#55

Earlier quoted context omitted.

> The FSF hired typists for me part of the day... That must have been a job from hell, probably went through a lot of them!

Manual transcription is still a thing today. Stenographers being one of the more well-known examples.

Stenotype keyboards are way better from a RSI point of view, though. Unfortunately, no one has come up with the "theories" or systems that would allow use of them for specialized input like programming.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#56
post #53
post #50

I don't know why, but I've often thought about the situation the Josh finds himself in. IE - having to find a different way of building software if I couldn't use a keyboard and mouse due to medical condition. The solution I always imagine is paying someone or having my employer pay someone to strong style pair program with me. Perhaps a student, junior developer or even someone unfamiliar with software development e…

> The solution I always imagine is paying someone or having my employer pay someone to strong style pair program with me Wow, that's some impressive out-of-the-box thinking! The OP's situation is something I worry about, as I'm worried my neuropathy will reach a point where I can't use my arms to type/mouse all day. I spent some time researching speech input for coding a year ago, and found a few fledgling solutions,…

I'm sorry to hear about your neuropathy, I'm glad my suggestion eases your mind!

On pairing in general: For a long period of time when I worked at bigger companies (before co-founding my current venture) I pair programmed 90% about of the time.

For the most part I really enjoyed it, we alternated pairs quite frequently and I worked with people at a range of seniorities. I really enjoyed alternating between playing the role of teacher vs learning from someone much better at the (insert programming language, database, business domain).

I think during that time I also had some of the most stimulating and deep conversations about software development. When pairing, pairs have a deep shared context that's hard to replicate in other scenarios.

There are some drawbacks though, the main one being it's extremely full on and emotionally tiring. I'd class myself as a high functioning introvert and after a day of pairing I needed some serious regenerative quiet time!

My advice to someone considering this suggestion would be to try some pairing now with friends or colleagues and get a feel for it and see if it works for your personality and working style.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#57
> I've heard that learning Vim can make this much more effective.

This doesn’t surprise me; Talon is a lot like Vim.

They’re both modal: Talon’s command mode roughly matches Vim’s normal mode, and its dictation mode roughly matches Vim’s insert mode.

Talon’s ordinals apply to Vim operations (normal mode) as well: operations can be preceded by a number which normally says how many times to do it. Talon’s “go left ninth” would be 9h or 9← in Vim. (h goes left one character.) “One zero third” is more cumbersome because in Vim it entails switching between modes (Talon and Vim modes don’t match precisely), but starting in normal mode, a13a0 works to append 1000 where the cursor is. (a to enter insert mode after the current position, 1 to type a 1, Escape to return to normal mode, 3 to say “do the next thing three times”, the next thing being appending a zero.)

Both Talon and Vim value this sort of consistency in being able to build bigger things from smaller pieces.

So I imagine Talon would work quite nicely with Vim, once you get talon to shift out of the way in some cases and let Vim handle those things. (It’s similar with browser extensions that let you interact with the browser Vim-style: when the page in consideration has extensive key bindings, especially Vim-like things like j/k for down/up, the extension actually gets in the way.) I expect you’d find that having used one made learning the other a good deal easier, since they’re working in just the same ways.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#58
Thanks for this! Sorry it had to come about as a result of injury/disease. Hopefully you're not going to hurt your vocal tract -- are there precautions you take against this?

My interest currently is in voice-assistants and developing something like a personal assistant / code-monkey / grad-student.

I did have a stroke a few years ago, from which I've recovered, but it made me realize that almost all of my work, hobbies, aspirations, are tied to fairly extensive hand-eye integration. It's good to start building a Plan B before I need it.

On a(nother?) side-note, my 88 year old dad still codes and his experiences have really shown me how bad we are at accessibility, as a developer community.

One final thing -- watching you work in this way highlights for me how low-level a lot of the stuff we do as developer is. There's a certain amount of just text massaging that seems irrelevant to what we're actually trying to get done (although it can be fun/soothing/aesthetic).

EDIT> The last point is why I can't give a fark about Vim/Emacs/whatever debates and obsessing. It seems like it might be fertile ground, but IMNSHO it's a trap.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#59

I love the eye tracking. Even without having any issues with my hands, I'd like to have quick mouse movements without having to take a hand off of the key board. Zooming in to have more precision is clever. It seems the current generation systems aren't working for Linux yet, but that gives me hope to have a way to work with it at some point in the future.

> It seems the current generation systems aren't working for Linux yet This isn’t correct. Talon supports eye tracking on all of Windows, Linux, and Mac with the same hardware.

I think parent is talking about the Tobii. I was also very very excited about it (I can’t find a mouse or trackball that won’t give me hand or shoulder or wrist pain) but then I saw it only works on windows... and nothing else. No Linux support. So that’s a no go.

If someone knows of such a device for Linux, I’d love to hear about it!

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#60
post #56
post #53

Earlier quoted context omitted.

> The solution I always imagine is paying someone or having my employer pay someone to strong style pair program with me Wow, that's some impressive out-of-the-box thinking! The OP's situation is something I worry about, as I'm worried my neuropathy will reach a point where I can't use my arms to type/mouse all day. I spent some time researching speech input for coding a year ago, and found a few fledgling solutions,…

I'm sorry to hear about your neuropathy, I'm glad my suggestion eases your mind! On pairing in general: For a long period of time when I worked at bigger companies (before co-founding my current venture) I pair programmed 90% about of the time. For the most part I really enjoyed it, we alternated pairs quite frequently and I worked with people at a range of seniorities. I really enjoyed alternating between playing th…

> I'd class myself as a high functioning introvert

Hah, I'm stealing that phrase to describe myself too!

I've never "formally" done pair programming, but have sat and coded/reviewed with colleagues for short stints, and generally found it OK.

From time to time I have to do full days (or several days) of workshops, and find those really tiring and emotionally draining. I always think I fare better 1-1 with people though, so if pairing was a necessity, I'm sure I could adapt.

Post reply on HN