Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

101–110 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#101
The eye tracking looks great, just as a regular additional way of using the mouse for things like focusing windows or tapping dialogs. I'm very tempted to get one of those.

Does anyone with experience of them know what the multi-monitor support is like? I have 3 screens side be side and one below, so quite a large range to cover that requires some head movement to go across all of them. Will it work with viewing over that wide of a space where I'm not always directly facing the primary screen?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#103

Appreciate all the discussion! I touch on this in the article, but I should say: I'm an edge-case when it comes to CTS. Most cases resolve spontaneously. Part of it is that my nerve dislocates when I bend my elbow, and this mobility causes a bunch of additional friction / inflammation. This happens naturally to about 13% of the population, and mine is particularly pronounced. All this to say, if you ever do start to…

What microphone are you using?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#104
Has anyone with joint pain tried accupuncture? Friend swears by it when she went to Tokyo, Japan to get it done a couple years back, 2018 if I recall correctly.

She recovered without taking any medication!

Edit: updated typo and extra info about not taking any medication.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#105
Reminds me of this story I encountered the other day. I have been on the same track, successfully too, for other types of pain (neck, shoulder and back pain). I post this here because this is an evidence based technique and think people should at least know that this is a possibility.

10/23/2008

"Hi all,

I have been battling computer related RSI that runs from elbows to fingers for 10 years. It started 2 weeks into my first software programming job out of college, and has been a constant disability since. I went through ergonomics, biofeedback, drugs, acupuncture, physical therapy, 5 jobs, Carpal tunnel surgery, a year of rest, voice Software, foot pedals, hands free mice, you name it.

I finally gave up software and changed careers 2 years ago to a more Sales focused job and still couldn't keep up with the typing. The only thing that has worked is having a full time computer assistant who sits with me and does what I say paired with a tablet PC that does handwriting recognition. Its dreadfully embarrassing in the workplace, slow, frustrating and just plain no fun. I got a rhythm though, got back on my feet and all went well for the last 2 years until a few months ago my throat began hurting (i talk all day now) and I was diagnosed with acid reflux.

With no arms and no voice, my future looks bleak and it has already been bad for almost a decade now.

I have officially given up on western medicine (my parents are doctors) and begun to explore alternatives. I stumbled on to Sarno and this newsgroup on Monday, read Sarno's book today. Does it really work? I mean, I have had non-stop, permanent, 24/7, life altering pain for 10 years. It has been my constant nemesis, battle, and focus.

It seems too good to be true.

I definitely have the personality traits, the onset of pain coincides with a rough first job out of college and my history of back spasms, shin splints and now reflux seems to fit the symptom imperative. I want to believe but can someone recover after this long of continuous computer pain?

-Dan"

source: http://www.tmshelp.com/forum/topic.asp?TOPIC_ID=5187

=====================

12/12/2008 (8 weeks later)

"Hi,

I had 10 years of 24 hour a day, 7 days a week chronic pain in my fingers, wrists, forearms, arms, elbows from typing, mousing, and using a computer up until 12/12/2008 as a computer science student and as a software engineer. I believed I would live with it the rest of my life. It took me 8 weeks to fully recover.

I am an interesting case because I don't have any major trauma's in my childhood or daily life that led to TMS but just a generally stressful family and a high achieving, perfectionist attitude.

I grew up with a solid family unit, a very close identical twin brother, I was good at sports, school, had high self esteem. I went to college and was the captain of the track team. I found computer science and loved it. Things were on the up and up.

My parents got divorced when I was in college but it didn't seem to bother me and transitioning to the working world. I was in a cubicle typing away and I didn't like it but I "sucked it up" and did my job the best I could. I started getting pain while typing/mousing and nothing ergonomic, or medical seemed to help. I plowed through the pain and kept working. I tried everything for 10 years. It was awful, bad, horrific and depressing all wrapped up in one. I finally gave up and left computers behind to be a project manager, and then a headhunter. But the pain was permanent and it still followed me no matter what job I did and how little I used a computer. My quality of life was pretty low. I found TMS 6-7 weeks ago and the recovery began.

I wrote an entry the day I found the TMS diagnosis and checked in along the way so the best way to learn my story is to read it below:

http://www.tmshelp.com/forum/topic.asp?TOPIC_ID=5187

I think I have a personality conducive to this type of pain defense mechanism. I am high-achieving, I have an intense and high achieving family, but I have a sensitivity to what others are feeling/thinking that governs much of my decision making.

I think the key is reading enough information (posts, books, etc) that you are 90% sure that this is possible and could be your diagnosis and then get ready to do some serious crying. Its a personal journey that ends really well and it starts with writing down or talking about all the crap that has ever happened in your life. I started with when my twin brother got surgery when we were 4 years old - my earliest memory - and ended with my boss telling me that the phone system I installed sucks. And it had 1500 items in between. I feel flushed out now and clean and ready to tackle the world. This newsgroup has been helpful and the process is hard but I honestly believed I would have to live with this my whole life, and now I don't. My phone number is at the end of my thread listed above. Call me, I would be happy to tell you about it live.

- dan"

source: http://www.tmshelp.com/forum/topic.asp?TOPIC_ID=5301

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#106

I have suffered with thoracic outlet syndrome for almost 2 years. Countless physical therapy and deep tissue massages helped alleviate the pain but nothing seems to solve the root cause. I am 26 and the thought of not being able to use my hands to do something I have frightened the hell out of me. That's when I stumbled upon John Sarno and his book "The mindbody prescription". I thought it's pseudoscience and I read…

100% think that all of us should know the technique that Dr. Sarno has discovered! I have had great success with neck, upper back/shoulder and lower back pain using the essence of what is going on here with pain.

The latest research I have had success with is based on Sarno's work and started with Dr. Schubiner's "Unlearn Your Pain" workbook and the "Unlearn Your Pain" podcast by Alan Gordon via Curable, plus all the Curable content has been GOLD for eliminating 90% of my pain so far. And I am at the tail end it it gets less and less every day (I can finally sleep through the night without having to move because of pain).

Life changing information here!!!

Some success stories using these techniques specifically on RSI and carpaltunnel are here:

https://www.tmswiki.org/ppd/Repetitive_Strain_Injuries_-_RSI

https://www.tmswiki.org/ppd/Carpal_Tunnel_Syndrome_(CTS)

And, the Seattle Repetitive Injury Support Team even states on their website > "we think we have finally found a cure for most RSI cases". http://www.satori.org/rist/

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#107

Earlier quoted context omitted.

I'm not so sure about this. My understanding is that RSI is caused by inflammation, and ignoring it will only make it worse, to the point of needing surgery. Although it sounds like you've been dealing with this longer, so I'll give you the benefit of the doubt

I see where you are coming from and I strongly suggest to try the conventional route to eliminate any serious physical problems before ending up here. My point is that RSI problems are not just physical but tightly coupled up with your mind. I wish someone can do a better job of explaining neuromuscular connection such that this won’t be pseudoscience anymore.

Unfortunately the conventional route can end up in unnecessary surgery and a hopeless cycle of disappointment. I would recommend trying this crazy-hard-to-believe (but is legit) route first, as there is no risk in learning about it and trying it. No drugs, no diet changes, no surgery etc. Just retraining the brain.

I recommend rsa25519 to give this a read, http://www.satori.org/rist/ (pasted below, keep in mind the cause is slightly out-of-date and not current thinking in this field, but the solution is the same)

> Welcome! The Seattle Repetitive Injury Support Team is a group of individuals who want to wipe out chronic Repetitive Strain Injuries (RSIs) such as tendinitis, thoracic outlet, and carpal tunnel. After many years of meeting to discuss treatment and coping approaches, share stories and ideas, listen to invited speakers and discuss good ergonomics, we are excited to announce that we think we have finally found a cure for most RSI cases! We know that sounds too good to be true, but please read on.

Over the years our members have tried just about everything to cure their RSI, yet we saw the same faces at meetings. People were finding coping mechanisms (massage therapy being the most recommended), but every month there we were, still in pain and not recovered -- and this went on for literally years.

Then, out of the blue, Nate McNamara contacted us about the approach he had used to cure his RSI problem. He graciously flew up to Seattle at his own expense to speak to our group. Here was someone who had been in the same situation as ourselves, and now was back to a normal, pain free life, even back working at a computer.

The essence of his approach was to recognize RSI as a form of TMS (Tension Myositis Syndrome). TMS, coined by Dr. John Sarno to describe common back pain, has a variety of possible physical symptoms. The theory is that the subconscious discovers that by reducing blood flow to various body parts it can cause pain and achieve its goal of suppressing unwanted emotions (through distraction). Sarno talks about suppressed anger, but stress and other related emotions can be a huge factor. Once you are in pain, the fear of becoming permanently disabled produces a vicious circle of fear and pain.

RSI probably starts with a case of physical overuse, which leads to pain. Normally the body would heal within about 48 hours, but the subconscious, by reducing blood flow to the areas involved prolongs this. The pain and tightness are real, but the cause is not physical but mental. It's important to stress that the sufferer is completely unaware that the subconscious is causing the problems.

As you may have guessed, the treatment is not physical, but involves foremost becoming convinced that you just have TMS and not a physical problem. This liberates you from your fear of hurting yourself further. Stopping all other treatment is also important to prove to yourself that you really believe in this solution. Various visualization exercises can help change your subconscious reactions (since you don't have direct control over these).

Amazingly, some people recover within a few hours, but most take a few weeks or months.

After Nate's talk, a number of us began to take this mind-body approach to our RSI problems. The current score to our knowledge is three cured and one doing much better! That may not sound like much, but to our knowledge, no other treatment has cured any of our members, although it is certainly possible they just never let us know.

We followed up on Nate's talk with videos and meetings on this topic, but it's now been more than a year and we feel that most of our members aren't taking these important steps to cure themselves, for whatever reasons.

Therefore we have decided to stop meeting, and instead focus on a mentoring program where those who are pursuing the TMS approach assist others directly who also want to follow this treatment philosophy.

If you are interested in seriously pursuing this approach, please contact us. To be honest, we aren't sure how good we can mentor, but we feel that it's the best way to get people cured.

To start this approach do as much research as you can to truly convince yourself that you have TMS. See the recommended background information below. The Amir book is highly recommended for it's visualization exercises.

All we have is years of personal experience to go on here, but we really believe we have found a solution to RSI. Our challenge is to figure out how to turn this into large numbers of cases cured or prevented. If you can help in any way please contact us!

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#108

Earlier quoted context omitted.

Manual transcription is still a thing today. Stenographers being one of the more well-known examples.

Stenotype keyboards are way better from a RSI point of view, though. Unfortunately, no one has come up with the "theories" or systems that would allow use of them for specialized input like programming.

I've seen a few conference presentations about using steno for programming. It does seem to involve building your own library of commands, but people seem to have done it and use it daily.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#109
post #90
post #64

Earlier quoted context omitted.

Am I the only one who just keeps their hands/wrists straight on a regular unsplit keyboard? I've always naturally done this and never had any problems with strain. The best way to describe it is that the home keys aren't ASDF, but more like WEFV with my forearm angled so it's straight from elbow to fingertip. This exaggerates the effect but demonstrates it clearly; the real home positions are more like intermediate p…

On my keyboard - which is.. I don't want to say 'full size', because maybe it's '80%' or whatever, but it's a regular external keyboard with numpad - f & j are centre to centre 60mm apart. If I outstretch my arms, straight, my index fingers are pretty much in line with my shoulders. So no, when I use my regular unsplit keyboard my hands/wrists are not straight, because my shoulders are not 60mm apart..? Am I missing…

I believe GP was talking about elbow-hand being completely straight. That would make sense if their elbows are on arm rests or some such thing.

It does sound interesting. I did type this comment trying it, and it was pretty comfortable, with the exception of my right wrist. That being said, I've got kind of weird seating set up.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#110

I love the eye tracking. Even without having any issues with my hands, I'd like to have quick mouse movements without having to take a hand off of the key board. Zooming in to have more precision is clever. It seems the current generation systems aren't working for Linux yet, but that gives me hope to have a way to work with it at some point in the future.

Keeping the hands on the keyboard would really increase productivity and reduce repetitive motion injuries. I know that editors like vim and emacs let you keep your hands on the keyboard, but interacting with anything else requires the mouse/touchpad.

At least 10 years ago, I imagined that the mouse could be replaced with something like a brainwave reader. You can train your brain to emit simple patterns that can be picked up either with a headband or remotely. It would take individual training, but with a few basic patterns, one should be able to do mouse movement and clicks. The eye tracking seems like the next best thing, especially since it is already implemented.

Post reply on HN