Live data from Hacker News

Cursorless – A spoken language for structural code editing

github.com

51–60 of 73 posts

Re: Cursorless – A spoken language for structural code editing

#51
post #42

Earlier quoted context omitted.

Talon has an eye tracking mouse, the necessary hardware runs around $250.

I would say that an eye tracking mouse is what OptiKey has, and for Talon it's more of an eye tracking+head pointing+voice clicking mouse. If you have tmj/vocal cords/neck problems you won't be able to use eye tracker as a mouse replacement.

> for Talon it's more of an eye tracking+head pointing+voice clicking mouse

> If you have tmj/vocal cords/neck problems you won't be able to use eye tracker as a mouse replacement.

I disagree with these assertions. There are several options in Talon for eye tracking. Optikey is a nice suite of functionality, but it's Windows-only, while Talon is cross platform. I also have a few users who use the Optikey UI but prefer to use Talon's eye tracking to control it. I've certainly recommended Optikey to users.

The updated direct control mouse mode in Talon 0.3 can be used without any head tracking at all as long as your targets aren't too small.

The zoom mode explicitly uses no head tracking as well.

To click or trigger the zoom mouse, you can make a pop noise, use voice commands, or a physical mouse button, or keyboard, or any keyboard emulating input device (such as a foot pedal), or use one of the dwell options.

Re: Cursorless – A spoken language for structural code editing

#52

Earlier quoted context omitted.

Thanks for the tip. Any reason run.sh isn't mentioned anywhere in the documentation? https://talonvoice.com/docs/

Isn't it mentioned in the README in the tarball?

The tarball, talon-linux.tar.xz, didn't come with a README, neither under root or /resources.

I could certainly have broken run.sh open in vim (and should have), I could have looked into /lib and found the missing dependencies, but given that there was a binary literally called talon, the general expectation is this is the binary to be linked to my PATH and run directly. Using a run.sh is not necessary when the same could be achieved by making the talon file a shell script and renaming the binary to something else.

Re: Cursorless – A spoken language for structural code editing

#53

To anyone tempted to get into Talon: note that it is very much not open source - https://talonvoice.com/EULA.txt (except when you contibute code to them; in that case you implicitly license your code as open source, but theirs is still their property).

Are you on Linux?

Re: Cursorless – A spoken language for structural code editing

#54

To anyone tempted to get into Talon: note that it is very much not open source - https://talonvoice.com/EULA.txt (except when you contibute code to them; in that case you implicitly license your code as open source, but theirs is still their property).

Are you on Linux?

Yes.

Re your deleted comment about "hostile forks": allowing users to run whatever software they want is precisely the point of open source. It's not "hostile" to have different opinions to you.

The project looks cool nonetheless, but it is still true that if the users don't control the software, the software controls the users.

Re: Cursorless – A spoken language for structural code editing

#55

Earlier quoted context omitted.

Are you on Linux?

Yes. Re your deleted comment about "hostile forks": allowing users to run whatever software they want is precisely the point of open source. It's not "hostile" to have different opinions to you. The project looks cool nonetheless, but it is still true that if the users don't control the software, the software controls the users.

I'll repost that in sibling. I just realized that I'd rather have people saying "ah, typical of proprietary software to not support Linux" than "your work has no value because I can't compete with you using your own code". I'm not going to stop supporting Linux because of this, but it still seems like a backwards incentive.

This is a business, not a pet project. I work on it full time and it is my meager source of income. I'm not convinced it is possible to make much of a living on open source.

I do mean hostile when I say hostile. I have a particular project of mine in mind, where the fork renamed, split the user base, and lagged behind my changes. They later just reset to my HEAD (as I was making way more progress than the fork), then removed attribution in the docs and solicited donations for the work I had done. I burned out on that project shortly after.

> if the users don't control the software, the software controls the users

Talon is free, runs offline, cares about user privacy and security, and gives an incredible amount of control to the user. The entire voice interface and behavior is defined in the user's code. There are several open source projects in this space and none of them are anywhere close to Talon on making it easy to build and customize your own voice / hands free interface.

Re: Cursorless – A spoken language for structural code editing

#56

To anyone tempted to get into Talon: note that it is very much not open source - https://talonvoice.com/EULA.txt (except when you contibute code to them; in that case you implicitly license your code as open source, but theirs is still their property).

> very much not open source

I contribute quite a bit to open source as part of my work on Talon, including the speech engine, the tools I use for model training and dataset creation, and I've upstreamed security fixes and other improvements to core projects like CPython and LibUSB.

> except when you contribute code to them

That clause isn't for code contributed to the Talon app, it's so other users in the community can use ad hoc scripts posted to Slack. Talon itself does not use any code under that clause and I still recommend users explicitly license their community code.

I've had the open source conversation plenty of times, but basically it boils down to: it feels like the bare minimum to protect myself from hostile forks given I have been working on it full time for over five years now, give it away completely for free, and provide high quality support.

It's also segmented so all of the primary functionality (e.g. voice commands and their behaviors) are implemented in open source user controlled code, and I open source plenty of the components of Talon.

You might read the handsfreecoding review [1] which covers this a bit.

[1] https://handsfreecoding.org/2021/12/12/talon-in-depth-review...

Re: Cursorless – A spoken language for structural code editing

#57
post #47

The issue i have with this and will always with speech to text software. my voice will never pick it up unless im very clear and i take the time to prounouce a word or a chord?. I will have more luck writing in a stenograph keyboard or a chording keyboard then this. I like the fact that this technology assits for those who can have their voice be read by a computer. It will never work for me... Despite how modern a l…

You may want to look at your hardware setup. Modern (1-3 year old) laptops now generally have incredibly good echo cancellation microphone setups which massively improve recognition accuracy. If you are stuck with an older machine, try a USB based (integral ADC, the logical inverse of a DAC used for playing audio) lapel mic. These can be had off amazon for a tenner, but will infuriate you everytime you forget to take…

thank you for the suggestion. i will try to look into it. it just at the same i was in highschool the technology wasn't as good for my voice. so i just learned how to type faster. but im willing to give it a shot and i can be a like "beta tester" or contributer if needed. i tend to use wired stuff since i do music production and am very picky on how stuff comes out when im finally mixing them.

Re: Cursorless – A spoken language for structural code editing

#58
post #45
post #5

Earlier quoted context omitted.

I think the main reason is too easily distinguish (to the program) between letters - and I guess to also have shorter words.

That's also the NATO phonetic alphabet's raison d'être, where a lot more's on the line; a lot more research time and money has been spent on it over the years. Air, Jury, and Quench in particular strike me as ambiguous examples NATO would never go for. Krunch is just baffling. Plex breaks the scheme followed by every other (that it's the first letter in the word).

Talon's alphabet is extremely carefully designed. There's de bruijn sequence style redundant phonetic information in each letter allowing the single syllable words to be mashed together at high speed without pauses (even allows for a bit of slurring to fit more than one letter into a syllable). I can type 60wpm with it, see me spelling a last name around 0:40 in this video for typical usage [1]. It's also carefully tested (with an American accent) to not jam up your mouth / tongue when you say the letters in any order.

You can remap it to whatever you want, and a few users choose either NATO or a custom alphabet. But most stick with Talon's with minor changes (e.g. they might change a specific word if it's harder to say clearly in their accent, or if their mic or environment makes it more difficult to pick up specific sounds).

You can also use the English alphabet for spelling anywhere you can say a word, e.g. "say hello" and "say H E L L O" both work the same.

This isn't a voice assistant where you belt out a command once in a while then move on with your day, and this isn't a low fidelity radio link. These words may be said tens of thousands of times a day. Using three-syllable NATO words is slow and exhausting. It's common for voice programming systems to define their own alphabet based on a truncated NATO alphabet. I went further because I wanted to be able to use vim and play DCSS at very high speeds with my key commands instead of binding a bunch of app specific custom commands.

[1] https://twitter.com/lunixbochs/status/1378159234861264896

Re: Cursorless – A spoken language for structural code editing

#59

The whole ecosystem around Talon[0] is fantastic. You'll probably want to start with [1] if it's your first time, it's a community-maintained talon config which is a good starting point for customization. The speech recognition engine built for it is great and the whole thing is free of charge, though I do recommend donating on Patreon [2]. Cursorless is a nice rethinking-from-the-ground kind of project. However, you…

I agree. Talon is incredible. Before I discovered it, I was convinced that my career would be ended early by RSI, as it was getting slightly worse every year. Meanwhile voice coding looked like impenetrable wizardry from the few videos that I had seen.

But after discovering it and getting through the learning curve (and there is one, that's probably the biggest reason why more people don't use it), it's something I look forward to using for the rest of my life. Even as my wrists start to gradually recover, being able to use voice/noise/eye tracking in tandem to the keyboard, allowing you to use the best for each purpose, is just so much more productive (not to mention, cool! fun!) that I doubt I'll ever go back.

Re: Cursorless – A spoken language for structural code editing

#60
post #48

Earlier quoted context omitted.

I have some very mixed feelings about talon. I looked into it a decent bit when I started having hand issues as well, and never got anywhere useful with it. For one, it needs to be programmed to do anything. This gets you all the options, which is wonderful, but also is a bit of a problem when you're looking into because you can't, ya know, program. And it's not a simple thing to interact with. Lots of moving parts a…

It sounds like Serenade.ai would be more useful to you. It is more of a complete product, with the ability to add custom commands on top of it. Plus it's open source. I'm not trying to steal the spotlight from Talon/Cursorless though, they are amazing and super useful. But it is definitely a much more fragmented ecosystem which requires more work.

Are people still maintaining Serenade? It looks like since they open sourced, it's mostly just been one person committing to it: https://github.com/serenadeai/serenade/commits/master
Post reply on HN