Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

Post reply on HN
Print view View original

31–40 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#31

Post by jedimastert »

Symbiote wrote: post #24

When I started my undergraduate degree, there was a PhD student everyone knew -- he looked like he ran the Linux User Group, which he did, he coded his own window manager, wrote games in Haskell etc. At some point, he got some hand injury that meant he couldn't type normally for months. He would only have been 21 or 22. This scared me, so since then I've followed the standard advice on avoiding computer-related injur…

I had a bit of a scare around the same time. I was following dual passions of computer science and piano playing, meaning around 12 hours a day of typing and/or piano-ing. I started to develop regular tendinitis in my wrists and fell down a rabbit hole of ergonomics and posture. With computers you can replace keyboards if you have money, but I was a musician in college. Also, you can't really change a piano keyboard all that much (and you really wouldn't want to) so I did a real deep dive into posture and getting the most out of what I already had

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#32

Post by codenesium »

I personally have had carpal tunnel release surgery and bilateral rib resection for thoracic outlet syndrome. These surgeries fixed most of my issues but for the last couple of years I haven't tolerated sitting well because of my back so I use a standing desk. I've thought about a computer interface that is the opposite of this article. Something that requires more non-repetitive physical movement so that you're exercising while you program. Sure it would take longer to type but we all know working on a computer all day is bad for our health.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#33

Post by joeseppy »

codenesium wrote: post #32

I personally have had carpal tunnel release surgery and bilateral rib resection for thoracic outlet syndrome. These surgeries fixed most of my issues but for the last couple of years I haven't tolerated sitting well because of my back so I use a standing desk. I've thought about a computer interface that is the opposite of this article. Something that requires more non-repetitive physical movement so that you're exer…

Sounds like an interesting idea for a VR/AR game. I'm picturing somewhere between fruit ninja and a vana white simulator

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#34

Post by thesuitonym »

Ottolay wrote: post #7
rco8786 wrote: post #4

The eye tracking sounds like it could be incredibly useful as a 3rd input. Very cool stuff, thanks for sharing

In addition to a pointer replacement, there is an idea of using eye tracking on a virtual keyboard with a swype type entry.

I don't know about that, but I'd love to have eye tracking for focusing windows. Since I don't generally maximize windows, I find that I'm often typing hotkeys for the window I'm looking at, not the window my mouse is on (or was last focused, depending on your wm)

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#35

Post by bdickason »

Not sure if this helps but I had a similar issue with my joints (arm, shoulder, wrist) where I would scream in pain when lifting a small book or anything above 3 lbs.

The issue was caused by my immune system acting up which would cause my tendons to become inflamed. After a year of misdiagnosis, Doctors found some heavy medication (methotrexate balanced with plaquenil) to help regulate it. One of the medicines side effects was listed as ‘death’ on the label.

After a year of experimenting, I found that major diet changes (lots more hearty greens, way less sugar and carbs, no caffeine or alcohol), improved sleep, reduced stress (quit my stressful job) completely alleviated my symptoms. I would still have flare ups from time to time which I reduced via physical therapy / exercise (to strengthen muscles supporting my tendons).

Just sharing as I had a somewhat similar condition and was surprised that the fix didn’t have to be a pill.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#36

Post by heinrichhartman »

> The biggest issue I've found so far is voice strain

My singing teacher started out as a voice therapist. He is still working with a lot of people with damaged voices that are there for therapeutic reasons as well as professional opera singers that there to level-up their game.

He says, the best thing you can do for your voice is to sing.

Singing will give you the proper "support" from your lungs and train the vocal muscles without wearing them out. One of the key aspects here is producing high quality vowels, and don't let the sound be disturbed by consonants. This is opposite to how the voice is used when speaking (in most languages), where consonants are of primary importance for people to understand what you are saying. However, consonants are interruptions in the air-flow and can easily wear the voice out and cause stain.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#37

Post by falcolas »

crispyambulance wrote: post #25
ihaveajob wrote: post #6

Earlier quoted context omitted.

It's pretty telling that RMS had to stop coding because CTS. Edit: Not carpal tunnel syndrome, as he explains in https://stallman.org/stallman-computing.html """In the mid 90s I had bad hand pain, so bad that most of the day I could only type with one finger. The FSF hired typists for me part of the day, and part of the day I tolerated the pain. After a few years I found out that this was due to the hard keys of my k…

> The FSF hired typists for me part of the day... That must have been a job from hell, probably went through a lot of them!

Manual transcription is still a thing today. Stenographers being one of the more well-known examples.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#38

Post by melling »

The author links to this other project, which I’ve never heard of:

https://serenade.ai/

He also references Tavis Rudd’s viral voice coding video, which is already 7 years old.

https://www.youtube.com/watch?v=8SkdfdXWYaI

The future is arriving slower than i would have guessed. I thought we’d be developing on the iPad using voice, gestures, and eye tracking by now.

Can I a least get “build and run” with my voice soon?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#39

Post by joubert »

bdickason wrote: post #35

Not sure if this helps but I had a similar issue with my joints (arm, shoulder, wrist) where I would scream in pain when lifting a small book or anything above 3 lbs. The issue was caused by my immune system acting up which would cause my tendons to become inflamed. After a year of misdiagnosis, Doctors found some heavy medication (methotrexate balanced with plaquenil) to help regulate it. One of the medicines side e…

Drastically reducing “added” sugars (which includes alcohol), whether end-user added or added as part of the industrial food “manufacturing” process does wonders for one’s health.

Reducing sugar intake reduces inflammation (https://www.health.harvard.edu/heart-health/the-sweet-danger...).

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#40

Post by Snitch-Thursday »

2Gkashmiri wrote: post #26

I have had an issue with dictation software... I want a dictation software to read me what it typed on screen. Right now the software expects you to check what it wrote and you have to necessarily keep looking at the screen. Call me lazy but I wish for a method where I could lay back, speak a sentence, wait for transcribing, listen to what was written and ahead I go.

when I was writing my last novel draft using nuance dragon 15, it had the 'read that' command which would read the last phrase I inputted, but that wasnt enough since I had a wireless headset that let me wander the house talking to the computer, but still needed to see the screen to know if it got the dictation correctly.

I ended up fishing out a cheap $120 portable walmart windows laptop 2 in 1 and VNCing it into the desktop so i could see the screen, carrying that around with me worked.

Post reply on HN