Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why is there no high quality method for voice control of a PC?

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 124 posts

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#111

Post by pokeyrule »

jiehong wrote: post #110
pokeyrule wrote: post #47

Earlier quoted context omitted.

creator of Cursorless here. Happy to answer any questions

Seriously cool project! This reminds me of easy motion for vim or ace-jump for emacs. Do you think it would be possible to have an on-demand contextual hat decoration? Like you say “show hats words” and only words get decorated with hats and you pick one. It would allow you to maybe show hats only on square brackets or only on function arguments, etc. I find the number of hats with colors a little bit hard to disting…

Thanks jiehong!

The reason that the hats are always present is that the way to code faster by voice than be keyboard is to speak fluently, minimising pauses, the way we speak regular human languages. If we had to say a command and then wait for the hats to appear, that would break the chain.

Re mapping, we use something called the "Command server", which allows us to use file-based RPC to run commands in VSCode. That way it is easy to send more complex commands, which are required by Cursorless

IntelliJ support is definitely one of the most requested features; once I'm done rewriting some of the core engine I'll probably take a swing at that. Here's the issue that tracks extracting cursorless into a node.js server so that it can be used by other editors: https://github.com/pokey/cursorless-vscode/issues/435

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#112

Post by boomka »

There are some tools, I think the reason they will never become widespread or high quality is that voice is just not a great medium for conveying that type of info in the first place. If I type a sentence and then decide to make a correction it is very difficult to explain in words but very quick to click and retype. If I want to position my window somewhere, I wouldn't even want to start thinking about how to explain it, I would just click and drag. And so on and so forth. This limits any potential markets for such tools greatly, so there is little economic incentive to develop them into anything truly high quality.

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#113

Post by daanzu »

newusertoday wrote: post #105
daanzu wrote: post #100

I have been coding entirely by voice for approximately 10 years now (by hand long before that). Most of that time I have been using the Dragonfly ( https://github.com/dictation-toolbox/dragonfly ) library to construct my own customized voice coding system. The library is highly flexible and open source, allowing you to easily customize everything to suit what you need to be productive. It is perhaps the power user an…

what sr engine do you use for your personal setup? is it kaldi?(assuming you helped developed it :-) ) .

Yep, I have been using my Kaldi backend through Dragonfly exclusively ever since I got v0.1.0 working.

I bootstrapped writing it initially using the Dragonfly WSR (windows speech recognition) backend, because that gave me the best accuracy out of the available options at the time. All of my development of it since the initial working version has been done using each previous version, so now it is basically bootstrapped itself. My productivity skyrocketed once I switched to Kaldi, due to being able to use my custom trained speech model just for my voice for orders of magnitude better accuracy, plus dramatically lower latency. (And it freed me from being dependent on closed software out of my control.)

I bootstrapped my personal speech model by retaining the commands from me using WSR. My voice is quite abnormal, and it took only 10 hours of speech data to train a model dramatically more accurate than any generic model I've ever used. And of course, I retain much of my usage now with Kaldi, so my model improves more and more over time. A virtuous flywheel!

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#114

Post by daanzu »

newusertoday wrote: post #109

I tried using talonvoice but the recognition engine failed to understad lot of words. I then searched for pronunciation of those words on google and tolonvoice detected them correctly. In the end i learned to pronounce the words in american english so that talonvoice can understand them ;-) .Not what i was hoping for, i wanted to teach computer to recognize my voice not the other way around.

With an open system/engine, you can train your own personal speech model. For kaldi-active-grammar (https://github.com/daanzu/kaldi-active-grammar), you can do so without all that much difficulty, although the process/documentation could certainly use improvement.

I bootstrapped my personal speech model by retaining the commands from me using WSR. My voice is quite abnormal, and it took only 10 hours of speech data to train a model orders of magnitude more accurate than any generic model I've ever used. And of course, I retain much of my usage now with Kaldi, so my model improves more and more over time. A virtuous flywheel!

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#115

Post by warrenm »

willcipriano wrote: post #51
fouc wrote: post #31

Earlier quoted context omitted.

I would imagine GPT-3 or similar would be able to fix replace the garbled 1 out of 20 words with something that actually make sense in context.

How do you know what word is garbled?

...and how do you know which word you meant (even if it's not garbled)?

The number of homonyms (and near-homonyms) in English in huge

It's been a major issue for some users of W3W (eg https://cybergibbons.com/security-2/why-what3words-is-not-su...)

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#116

Post by warrenm »

mikob wrote: post #85
mertd wrote: post #6

As far as human computer interfaces go, keyboard and mouse probably win comfortably in both bandwidth and latency against speech to text in almost all tasks. Former also requires a less physical effort and is creates less noise for others. My guess is that this shrinks the demand for good quality voice HCI significantly and those who really need it end up being overlooked.

You're limiting your thinking to the paradigm of visual interfaces paired with a mouse and keyboard. When all you have is a hammer... Here's some examples where bandwidth and latency wins with speech: 1. "Play here comes the sun" vs. opening spotify, waiting, clicking the search box, typing here comes the sun, pressing enter, waiting, scanning the page and clicking the right song. 2. "Send email to John asking him if…

> 2. "Send email to John asking him if he would like to Play golf"

Which John? Which of that John's contact points you have saved?

..and why don't you have the keyboard shortcuts for those actions committed to muscle memory by now?

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#117

Post by sapiol »

nosianu wrote: post #27

Voice input is good for high level tasks and goals, requiring a high level comprehension. For detailed work though the more direct method of translating movements is far more efficient. When you can describe an abstract end goal voice is great. When you have to actually do all the individual steps towards some high level goal then it's like telling a newbie programmer through some high level database optimization. Yo…

Offtopic: Hi, I saw your comments in some older thread about chelation (Cutler Protocol). I too am from germany and have some questions about your chelation protocol. Unfortunately, I can't reply anymore on that other thread. Can you contact me at 1u3_2d227vh7iadt@byom.de ?

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#118

Post by sapiol »

nosianu wrote: post #27

Voice input is good for high level tasks and goals, requiring a high level comprehension. For detailed work though the more direct method of translating movements is far more efficient. When you can describe an abstract end goal voice is great. When you have to actually do all the individual steps towards some high level goal then it's like telling a newbie programmer through some high level database optimization. Yo…

Offtopic: Hi, again. That didn't work as there is a 30m time-limit on byom.de. Can you please contact me again, but this time here: D-8ynpb9p087ukef2v@maildrop.cc

Re: Ask HN: Why is there no high quality method for voice control of a PC?

#119

Post by 4midori »

warrenm wrote: post #12

>The market is huge Apparently ... it's not Or, rather, it's not YET "huge" Sure - half the planet is online, but they're speaking myriad languages in more combinations of enunciation, dialect, and accent than is probably even calculable >the Natural Language Processing of "OK Google" and Siri are quite refined at this point Totally different to ask for today's weather and to tell a computer what to do - just like it…

OP here. Great comments and ideas, all. A few notes: * Talon is pretty great * I think the market for text to speech and voice control is huge, and maybe Dragon/Nuance rules it because of their patents, but oh, does it suck. Like being stuck on Windows 95 or something. * Voice Recognition is in fact currently good enough to get real work done efficiently * Serious RSI can't be fixed with ergonomics or better devices * If there were a modern alternative to Dragon, it would solve a chunk of the problem

It's true that computer control currently requires a lot of customization, but I see no practical reason why we can't at least make simple commands fast and accurate, i.e., 'create new html document in VS Code'.

Post reply on HN