Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

71–80 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#71

Earlier quoted context omitted.

I dunno... "swyping" your eyes around a virtual keyboard seems like it'd cause a lot of strain on your eyes.

There are alternative virtual keyboard styles (typically made for use with a mouse and impaired motor control) which would work better. Fewer "fine" movements. Combined with predictive typing, and you could go pretty far.

Do you have any examples?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#73
post #62

I’ve been having similar issues lately, and after looking at talon, serenade, and caster, I ended up using caster [0]. The different programs all have significant differences in usability, and have clever ideas behind them, but unfortunately automatic speech recognition is still bad enough that the primary factor is which has a better ASR engine. Caster supports Dragon Naturally Speaking, which is expensive, but enou…

Do you know of any workflows like this using entirely open source software? EDIT: Seems like caster itself has instructions for an open source recognition engine to pair with it. Not sure how accurate it'll be but I'm going to give it a shot! I hate the idea of relying on closed source software to be able to continue in my profession. If this works I'm definitely going to be donating to the FOSS options

My understanding is that Talon is built off of wav2letter's inference framework for ASR.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#74
post #39

Not sure if this helps but I had a similar issue with my joints (arm, shoulder, wrist) where I would scream in pain when lifting a small book or anything above 3 lbs. The issue was caused by my immune system acting up which would cause my tendons to become inflamed. After a year of misdiagnosis, Doctors found some heavy medication (methotrexate balanced with plaquenil) to help regulate it. One of the medicines side e…

Drastically reducing “added” sugars (which includes alcohol), whether end-user added or added as part of the industrial food “manufacturing” process does wonders for one’s health. Reducing sugar intake reduces inflammation ( https://www.health.harvard.edu/heart-health/the-sweet-danger... ).

Can confirm. Cut out almost all sugar from my diet. I've seen dramatic improvements in my health with the most significant being the elimination of brain fog.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#75
post #3

As a developer who also has cubital tunnel syndrome, having keyboards that places commonly use modifier keys [0] on your thumbs, such as Kinesis advantage or Ergodox, helps reducing the fatigue and symptoms immensely. [0] it is called Emacs pinkies for a reason, and yes, my CTS is totally emacs’s fault as well.

ye, modifier keys suck. This is why I prefer snake-case over the other casings because it needs no modifier key.

What languages do you use?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#76

Earlier quoted context omitted.

There are alternative virtual keyboard styles (typically made for use with a mouse and impaired motor control) which would work better. Fewer "fine" movements. Combined with predictive typing, and you could go pretty far.

Do you have any examples?

Not the OP, but Dasher is a classic example that works very well for a single 2D input like using a joystick: https://youtu.be/0d6yIquOKQ0

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#77
post #64

Earlier quoted context omitted.

The planck keyboard is a great keyboard, but possibly on par with the worst in terms of ergonomics. Ergodox, Diverge, Keyboard.io, really any split key keyboard does worlds for keeping hands straight and are fully programmable to very high degrees.

Am I the only one who just keeps their hands/wrists straight on a regular unsplit keyboard? I've always naturally done this and never had any problems with strain. The best way to describe it is that the home keys aren't ASDF, but more like WEFV with my forearm angled so it's straight from elbow to fingertip. This exaggerates the effect but demonstrates it clearly; the real home positions are more like intermediate p…

Yeah, same. I find keeping my fingers on the home keys makes me twist my wrists outwards, which gets uncomfortable pretty fast. I mostly try to minimise wrist action in general.

I've actually had a Planck EZ for a few weeks now and I agree it's probably not very ergonomic for the classic qwerty touch typist, or at least not much better than a normal keyboard barring the programmability.

But since I don't do that anyway I find the keyboard to be pretty nice in terms of customisability and avoiding stretching.

In general I feel my hands are used most naturally in close proximity to each other (at roughly abdomen height) so I'm drawn to small keyboards with lots of modifier keys. A spherical keyboard would be pretty interesting to try out.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#78
This might be able to be a bit faster if you use a virtual QWERTY keyboard displayed with eye tracking acting as a cursor, with select being a consonant, preferably /d/. You can map certain phonemes for other shortcuts, including one for click/right-click/vim-commands.

Articulating words takes too long, even subvocalization is, which has the advantage of only using non-voiced phonemes.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#79
> Honestly, it's just been such a relief to discover that my hands aren't needed for me to do my work.

This describes my reaction exactly. I'm noticing my rsi getting better, which I'm grateful for, but the gift of not being so scared feels like the bigger deal.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#80
post #65

Earlier quoted context omitted.

A few years back I started buying disability insurance, in addition to paying more attention to my ergonomics. My greatest economic asset is the years left in my career. For ~$1500/year I insure that. It's not cheap, but it would pay out until I was 65 at some reasonable percentage of my normal salary in the case I was severely injured, or otherwise unable to work. Anywhere from an ergonomics caused hand/arm injury,…

Not sure how old you are but I imagine over the next couple of decades we’ll eventually get more advanced computer interfaces. Gestures, voice, eye tracking, ... Being a software developer could be a job for those who physically can’t do other jobs. Subtle input gestures with Soli would be interesting. https://atap.google.com/soli/

Assuming physical injury (like RSI or trauma), yes, it's getting easier and easier.

But there are many other ways you could be injured and unable to work. My straw-man injury when I bought the policy was: "I'm a new skier, which is famous for major head trauma". Similarly, cancer, which directly or via treatment, causes exhaustion is another example of disability that is not solved with "if only I could just input text easier".

I'm glad there's so much work in accessibility going on, and that hardware, machine learning, and social awareness has gotten widespread enough to let non-huge companies dive in and make real progress. It's important to open the field to as many people who we can.

I just want to be sure people don't rely on technology being advanced enough at the exact time you may need it. You can easily get unlucky, and insurance exists to pay somebody else to take the risk.

Post reply on HN