Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On Voice Coding

dusty.phillips.codes

Post reply on HN
Print view View original

81–90 of 186 posts

Re: On Voice Coding

#81

Post by jedberg »

est wrote: post #69

Off topic, but I really wish to find out any Alexa-like "smart speakers" capable of voice programming. For example: 1. I would like to command the speaker listen for a keyword like the Fizz Buzz Test[1] if I counted to certain number. 2. Ask the speaker to remind me of something when hearing certain topics during a conversation. Much like the "if" keyword in text based computer programming languages. 3. Program a poe…

I think all the big smart speaker makers would tell you to just write a serverless function and hook it up to your speaker. It's unlikely they would ever create such a functionality.

Most likely if you want to make it work you'd either have to build your own smart speaker or make a serverless function that used one of the other voice programming programs mentioned in this thread as it's backend.

Re: On Voice Coding

#82

Post by lunixbochs »

est wrote: post #77
lunixbochs wrote: post #72

Earlier quoted context omitted.

I would probably start with the seeed studio ReSpeaker array hardware wise. Otherwise, can you write up a more specific use case list with specific commands and responses? This sounds fun and I can probably help you make this happen.

> write up a more specific use case list with specific commands and responses "no swear words" rule for kids. The speaker need to maintain a global counter for kid_id/word counter for every time a swear word is heard. Will reset every week for rewards/penalty, etc. The parents (as admin) could add/remove the swear words list, and can configure how long the counter need to be rotated. A non-trivial example, I am an ES…

Ok, here are some quick musings, I’m open to talking about different approaches for any of this or additional features I didn’t cover here. I’m hearing two stronger use cases here.

1. “Family database” - you tell it facts and it tries to remember and reproduce them later when asked. Like storing your family tree somehow and asking about it. This has a lot of nuance but is likely possible in some way with current technology.

2. Teaching assist. You give it a task, such as an English text, and it helps train and evaluate your kid.

For (1), I think there are three obvious approaches to me:

1. Manually enter the data with a computer, but use natural language queries (some kind of Intent model) to access it

2. Have a strict set of data that you can enter (such as family tree, grocery lists, voice memos), and allow slightly strict natural language-ish statements to add data, and natural language queries to access it.

3. Convince a general understanding model like GPT-2 to learn your facts, and just pipe your questions directly into it. This is the coolest answer but would likely also be wrong more often.

I think (2) is an easier task, and likely would use an entirely different approach for it than (1).

(2) is especially easiest if the system already knows the text of the poem. If you’re making up a new poem, or speaking a poem it can’t somehow look up, it will be harder to match two voices against the text of the poem.

One caveat to all of this is I’ve heard speech recognition doesn’t work as well on children because they aren’t well represented in the training data.

Re: On Voice Coding

#83

Post by est »

jedberg wrote: post #81
est wrote: post #69

Off topic, but I really wish to find out any Alexa-like "smart speakers" capable of voice programming. For example: 1. I would like to command the speaker listen for a keyword like the Fizz Buzz Test[1] if I counted to certain number. 2. Ask the speaker to remind me of something when hearing certain topics during a conversation. Much like the "if" keyword in text based computer programming languages. 3. Program a poe…

I think all the big smart speaker makers would tell you to just write a serverless function and hook it up to your speaker. It's unlikely they would ever create such a functionality. Most likely if you want to make it work you'd either have to build your own smart speaker or make a serverless function that used one of the other voice programming programs mentioned in this thread as it's backend.

> build your own smart speaker or make a serverless function

If I build my own smart speaker it would surely be run on my home server. Not so much for server-less. Yeah but I get it. the voice commands should be counted as new "keywords" or "functions". Let there be a general "voice programming" language.

Re: On Voice Coding

#84

Post by lunixbochs »

AstralStorm wrote: post #41

This article could use a better title. Voice coding makes me think of audio compression for voice. This is programming using voice recognition instead.

A small nitpick when talking about voice programming is there’s a superset of “voice control”. When your hands are hurt you don’t just want to type in your editor and terminal, there’s chat programs, application switching, web browsing, and other tasks it would be nice to offload from your hands too.

So the clearest way to represent some of this may be to encode the meaning of “voice input/control capable of programming” in a title. We might need a new name for this kind of input to best represent it.

Re: On Voice Coding

#85

Post by jedberg »

lbj wrote: post #49

Just a quick shout out to Microsoft (no affiliation) - Their Sculpt keyboard has taken all the discomfort out of 15hour coding sessions. If you're a professional developer, get one.

I've been using the Microsoft Natural Ergo for 20+ years. It's the very first thing I ask for when I get a new job, or if they don't supply keyboards, the first thing I bring from home.

I've never had any RSI type symptoms or even fatigue after long typing sessions.

The sculpt seems to just be a fancier wireless version of the same thing (although I haven't tried it so I could be wrong).

Re: On Voice Coding

#86

Post by woodson »

lunixbochs wrote: post #33
qixxiq wrote: post #23

It's still surprising to see Dragon Speech Recognition as the recommended (and only) choice here. Is anyone working on decent speech recognition for Mac/Linux or know good resources for that? The ideal output is a stream of what could have been said, as well as some alternatives, each with a confidence. Every alternative I've tried has not been as effective as the version of Dragon I used from 2011. I think the focus…

The Talon beta ships with wav2letter and a really good many-accent English model that can handle both arbitrary commands and free form English. All of my trained models and some information is posted here: https://talonvoice.com/research/

Are you going to release the speech data you collect at https://speech.talonvoice.com/ or is it proprietary?

Re: On Voice Coding

#87

Post by shrimp_emoji »

keymone wrote: post #25
ulisesrmzroche wrote: post #18

Earlier quoted context omitted.

Or you could take a short break every so often What we really need are better programmers

Tools matter. First ascent of Eiger’s north face took few days and team of 4. Last I heard Ueli Steck did it few years ago solo in less than 2 hours. Maybe he was better but not by that much. His tools were unquestionably better.

If better tools also make "worse" programmers (i.e., people saner and happier and uninitiated with the brain death rigors of struggling with awful/no tools), tools also matter more the better they get.

Re: On Voice Coding

#88

Post by gleb »

lunixbochs wrote: post #46
dwiel wrote: post #21

I've been voice coding for about 5 years now. For those of you not on windows, I use talon voice on mac (linux version is in beta). It works quite well and I'm at least as productive writing code by voice than I ever was by hand. I was someone who would spend the time to get my emacs and then later vim configs highly optimized, but there is something liberating about not constraining yourself to key bindings. I used…

At this point all of the projects mentioned in this thread (caster/talon/serenade) have some option for supporting the three main (win/lin/mac) platforms.

How do you get/request access to Talon on Linux?

Re: On Voice Coding

#89

Post by lunixbochs »

woodson wrote: post #86
lunixbochs wrote: post #33

Earlier quoted context omitted.

The Talon beta ships with wav2letter and a really good many-accent English model that can handle both arbitrary commands and free form English. All of my trained models and some information is posted here: https://talonvoice.com/research/

Are you going to release the speech data you collect at https://speech.talonvoice.com/ or is it proprietary?

I don’t consider it proprietary. As per the agreement I specifically ask for an open license so I will be able to release it in the future.

Right now it’s about 5 hours total, which isn’t a ton for actually training on, which is why I haven’t prioritized releasing it and haven’t even trained on it myself yet. I’ve been mostly using it for evaluation so far.

If someone approaches me and says “I have a compelling need for a bit of training data in the form of your prompts” I’ll probably prioritize a release higher.

As another perspective, a majority of the people at this point submitting their voice are already using Talon and just want the engine to be more robust.

Re: On Voice Coding

#90

Post by vl »

ijidak wrote: post #4

FYI, I thought my programming career was over due to RSI. Now, I only type while wearing long-sleeves. And of course, I still have to take regular breaks. I no longer suffer RSI symptoms. I'm guessing because it increases blood flow to the area and perhaps the warmth helps keep ligaments and muscles flexible and loose. Simple solution, but took a while to figure out. Hopefully this helps someone reading this.

For me it’s Kinesis Advantage and 3M wrist braces. This way I can type and mouse all day, otherwise I get pain in couple hours. For laptop keyboard braces help, but in general I avoid truly prolonged typing on laptop.

The key is not to let pain develop, stop immediately and develop solution, otherwise you can get through the point of no return.

Post reply on HN