Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hands-Free Coding: How I develop software using dictation and eye-tracking

joshwcomeau.com

Post reply on HN
Print view View original

91–100 of 141 posts

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#91

Post by lunixbochs »

ianhorn wrote: post #89
lunixbochs wrote: post #86

Earlier quoted context omitted.

Talon can also use Dragon remotely from a Windows machine or VM, kind of like aenea but more tightly integrated (the host is driving instead of the guest). And I do have some users voluntarily switching from Mac Dragon to the improved speech engine in the Talon beta. Mac Dragon has been kind of buggy for the last few years so you're not missing much.

Any chance you have pointers on how to set that up? You'd probably laugh/cry to see my setup right now, with my Windows desktop on the left monitors and my MacBook on the right monitors because I need both... purely because Dragon is only sold on Windows since this started been an issue for me. A more tightly coupled super-aenea sounds pretty fantastic.

Sure: First run Talon on both sides. Then go to your talon home directory (click talon icon in tray -> scripting -> open ~/talon). There's a draconity.toml file there.

On the Dragon side, you need to uncomment and update the `[[socket]]` section to listen on an accessible IP.

On the client side (Mac in your case), uncomment / update the `[[remote]]` section to point at your other machine.

You also need to make sure both configs have the same secret value.

From there, restart Dragon and look in the Talon log on the Mac side for a line like "activating speech engine: Dragon".

To prevent command conflicts, I recommend setting Dragon to "command mode" (if you have DPI), and only adding scripts to Talon on the Mac side.

If it doesn't work, you can uncomment the `logfile` line in draconity.toml on the Dragon side, restart Dragon, and look in the log to see what's going on.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#93

Post by lunixbochs »

chrismorgan wrote: post #57

> I've heard that learning Vim can make this much more effective. This doesn’t surprise me; Talon is a lot like Vim. They’re both modal: Talon’s command mode roughly matches Vim’s normal mode, and its dictation mode roughly matches Vim’s insert mode. Talon’s ordinals apply to Vim operations (normal mode) as well: operations can be preceded by a number which normally says how many times to do it. Talon’s “go left nint…

I developed much of the original Talon code using a single Python file (the combination of std.py + basic_keys.py) and Vim.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#95

Post by bdickason »

slashblake wrote: post #44
bdickason wrote: post #35

Not sure if this helps but I had a similar issue with my joints (arm, shoulder, wrist) where I would scream in pain when lifting a small book or anything above 3 lbs. The issue was caused by my immune system acting up which would cause my tendons to become inflamed. After a year of misdiagnosis, Doctors found some heavy medication (methotrexate balanced with plaquenil) to help regulate it. One of the medicines side e…

Are you me?

Maybe we're the HN equivalent of Fight Club?! Am I Brad Pitt or Edward Norton

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#96

Post by fao_ »

> In English, an ordinal number is one used to describe order, like "fifth" or "ninth" or "three hundredth". In Talon, they're used to repeat commands. If I wanted to go left by 9 spaces, I would say go left ninth.

Right! Like 9l in vim, but swapped.

I've always thought vim-like interfaces would be a good way to structure vocal input and I'm glad there are solutions operating in that space.

Edit: beaten to the punch :)

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#97

Post by lunixbochs »

ianhorn wrote: post #87
cybwraith wrote: post #67

Earlier quoted context omitted.

Do you know of any workflows like this using entirely open source software? EDIT: Seems like caster itself has instructions for an open source recognition engine to pair with it. Not sure how accurate it'll be but I'm going to give it a shot! I hate the idea of relying on closed source software to be able to continue in my profession. If this works I'm definitely going to be donating to the FOSS options

Yeah there is the Kaldi backend for caster, which I've tried on my Mac machine, since Dragon isn't a thing on Mac. Unfortunately it's not nearly as good :-( I'd like to try to record my usage of Dragon that I could fine-tune my own model, but it's harder to get around to hobby coding projects like that now that coding is more of a pain in the ass. The unfortunate reality in my case is that using Dragon is the least f…

Fwiw talon has a setting to record everything locally, annotated with the recognized words. In the next release it will also work when using dragon.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#98

Post by adamnew123456 »

fctorial wrote: post #75
Quarrelsome wrote: post #52

Earlier quoted context omitted.

ye, modifier keys suck. This is why I prefer snake-case over the other casings because it needs no modifier key.

What languages do you use?

Also along those lines: what keyboard layout? I've never seen a (QWERTY) layout where underscore isn't a shift-level key.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#99

Post by reanimus »

This is really, really interesting -- I've often wondered about how the accessibility space is meeting the needs of disabled programmers, especially after I briefly lost my vision due to an eye injury.

The eye-tracking setup in particular looks really nice. Even if I stuck to a regular keyboard flow I feel like that would be incredibly useful in reducing strain.

Re: Hands-Free Coding: How I develop software using dictation and eye-tracking

#100

Post by rohith2506 »

rsa25519 wrote: post #85
rohith2506 wrote: post #83

I have suffered with thoracic outlet syndrome for almost 2 years. Countless physical therapy and deep tissue massages helped alleviate the pain but nothing seems to solve the root cause. I am 26 and the thought of not being able to use my hands to do something I have frightened the hell out of me. That's when I stumbled upon John Sarno and his book "The mindbody prescription". I thought it's pseudoscience and I read…

I'm not so sure about this. My understanding is that RSI is caused by inflammation, and ignoring it will only make it worse, to the point of needing surgery. Although it sounds like you've been dealing with this longer, so I'll give you the benefit of the doubt

I see where you are coming from and I strongly suggest to try the conventional route to eliminate any serious physical problems before ending up here. My point is that RSI problems are not just physical but tightly coupled up with your mind. I wish someone can do a better job of explaining neuromuscular connection such that this won’t be pseudoscience anymore.
Post reply on HN