Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

Post reply on HN
Print view View original

131–140 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#131

Post by lunixbochs »

navels wrote: post #128
lunixbochs wrote: post #69

Earlier quoted context omitted.

Let me be more specific. The Tobii 4c, PCEye Mini, and Tobii 5* work approximately the same on Linux, Mac, and Windows with Talon out of the box. * The Tobii 5 does work, but is a work in progress and has some caveats until I do more work on it.

Would you mind elaborating on this? I'm ready to buy a Tobii but would like to know what the model 5 caveats are and how long you think it'll be WIP. And if you think the improved specs on the 5 make it worth waiting for. Thanks!

The caveats make it short term worse than the 4C for some mouse modes, long term it should be the best option overall once I fix some things.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#133

Post by cybwraith »

ianhorn wrote: post #87
cybwraith wrote: post #67

Earlier quoted context omitted.

Do you know of any workflows like this using entirely open source software? EDIT: Seems like caster itself has instructions for an open source recognition engine to pair with it. Not sure how accurate it'll be but I'm going to give it a shot! I hate the idea of relying on closed source software to be able to continue in my profession. If this works I'm definitely going to be donating to the FOSS options

Yeah there is the Kaldi backend for caster, which I've tried on my Mac machine, since Dragon isn't a thing on Mac. Unfortunately it's not nearly as good :-( I'd like to try to record my usage of Dragon that I could fine-tune my own model, but it's harder to get around to hobby coding projects like that now that coding is more of a pain in the ass. The unfortunate reality in my case is that using Dragon is the least f…

> I do like that caster is open source where talon isn't, but preferences like that are pretty low priority when say your career is on the line.

For sure if its on the line right now you use what is available and works best right now, but the reality of niche software is the companies/people backing it tend to close down or move on as they realize the market is too small to sustain them. Or Microsoft updates its OS in a way that forces them to expend considerable time making it functional again and they just don't have the bandwidth for it so you're stuck for some amount of time or indefinitely.

If you have no other choice, its definitely better than nothing and you want to use what enables you to do your job the best. But in terms of where I'd be willing to throw my financial support? It'd need to be something for the better of the coding community as whole. For me, this means an open source tool or set of tools, not a proprietary system.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#134

Post by kybernetikos »

zozbot234 wrote: post #55
falcolas wrote: post #37

Earlier quoted context omitted.

Manual transcription is still a thing today. Stenographers being one of the more well-known examples.

Stenotype keyboards are way better from a RSI point of view, though. Unfortunately, no one has come up with the "theories" or systems that would allow use of them for specialized input like programming.

Plover is often used for this, e.g. https://www.youtube.com/watch?v=jRFKZGWrmrM

http://www.openstenoproject.org/plover/

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#135

Post by codebje »

g0ggs wrote: post #119
codebje wrote: post #118

Earlier quoted context omitted.

I had pain and weakness in my right wrist for years, and also had many rounds of misdiagnosis. It turned out to be a benign tumour. Surgery was my answer, but it's only a partial one. It's a soft tissue tumour, the surgeon didn't expect to get all of it, and it's growing back now. It's been about seven years, and I expect I'll need surgery again within seven more years to be able to do simple tasks like lift objects…

Would this happen to be a schwannoma? If so, this was the case for me as well. Took a long time and a lot of sleepless nights before an mri found it.

Hemangioma in my case. I had an MRI but in the end it was a surgeon with the right experience looking at an earlier ultrasound that diagnosed it.

I had trouble believing he'd diagnosed it when he glanced at the ultrasound and calmly said what it was. Given I'd had medical practitioners send me for neurological studies to check it wasn't all in my head, suggest fusing my wrist to stop the pain, tell me there was nothing visible in X-ray, MRI, or the same ultrasound report the surgeon diagnosed me from... It seemed unlikely anyone could do an apparently simple diagnosis like that.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#136

Post by DantesKite »

peanutz454 wrote: post #116
DantesKite wrote: post #112

Earlier quoted context omitted.

I once had a very severe case of insomnia I thought was caused by some medication. It was such a difficult time for me, and after going through so many different solutions, I eventually discovered a book written by a Harvard doctor that had successfully stopped insomnia in hundreds of patients over several years. Basically, he discovered that most insomnia is caused by belief. It becomes a vicious cycle that feeds in…

Umm... book name would help.

Apologies. Here’s the book.

Say Good Night to Insomnia: The Six-Week, Drug-Free Program Developed At Harvard Medical School https://www.amazon.com/dp/0805089586/ref=cm_sw_r_cp_api_i_hS...

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#137

Post by Firehawke »

The one major problem with these solutions is how expensive it all is. If you don't find out about these programs BEFORE your finances get destroyed by health issues (both in cost of lost work time and in the medical care; most insurance will only cover SOME of the medical bills), you'll never be able to afford the hardware and software to work around them.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#139

Post by nshm »

lunixbochs wrote: post #88
whimsicalism wrote: post #73

Earlier quoted context omitted.

My understanding is that Talon is built off of wav2letter's inference framework for ASR.

I do not use the wav2letter@anywhere inference frontend - I trained the acoustic model using the facebook upstream code, but the decoder is almost entirely new, and on Windows I use Pytorch for inference. Talon ships with a libw2l.so/dylib on Linux/Mac built from my open source repos.

Feels like wav2letter will not be actively developed anymore. Understandable since it is hard to compete with Pytorch with a custom NN toolkit. Any plans to move to Pytorch/Tensorflow?

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#140

Post by psychometry »

quicklime wrote: post #8
psychometry wrote: post #5

Seems like it would be a useful feature in voice recognition to have a "whisper" mode to prevent strain.

I once had laryngitis for about a week, and during this time found out that for most people whispering puts more strain on the vocal cords than regular speaking: https://www.nytimes.com/2011/02/08/health/08really.html But a whisper mode might prevent ear strain for those working around you :)

Hmm, I take it back then. Maybe a sub-vocalization mode then...
Post reply on HN