Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is there any work being done in speech-to-code with deep learning?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#31

Post by mk4 »

raidicy wrote: post #30
mk4 wrote: post #12

Not coding - but openai's API in beta has a speech to bash function https://openai.com/blog/openai-api/

thank you I ended up signing up for the waitlist I saw the demo for the English to bash one shot learning if I could somehow use that and combination with a speech recognition software I might be able to achieve something along the lines said I'm looking for

Not to be too forward - but maybe ask then on twitter - there are a few conversations around accessibility and perhaps you could give them feedback from your needs.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#32

Post by xenonite »

Is the hackernews process broken? I currently see three comments being downvoted without any apparent reason: https://news.ycombinator.com/item?id=23507041 https://news.ycombinator.com/item?id=23506992 https://news.ycombinator.com/item?id=23507486

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#34

Post by daanzu »

bmc7505 wrote: post #22

Shameless plug, but I have been working on an open source IDE plugin [1] for the IntelliJ Platform which attempts to do this. Previously, we used an older HMM-based speech toolkit called CMUSphinx [2], but are currently transitioning to a deep speech recognition system. We also tried a number of cloud APIs including Amazon Lex and Google Cloud Speech, but they were too slow -- offline STT is really important for low…

I agree with everything you said, but I would add that a critical component of voice command and control is strict grammars. There is so much structure and context in what we speak, and being able to limit what can be recognized to only what can be reasonably spoken (based on the current context) can allow massive increases in accuracy. (EDIT: ah, you edited to add a mention of this as well.)

And one shameless plug deserves another! Vosk is a great project, but my kaldi-active-grammar [0] (mentioned in another comment here) also uses the same Kaldi engine, but extends it and is designed specifically for this use case. It supports defining many grammars, in any combination, and activating/deactivating them at will instantly per-utterance. I think it's probably a better fit as a backend for your project than vosk. My work focuses on the backend technology, so it would be great to have more front ends using it to put it within users' reach (so to speak).

[0] https://github.com/daanzu/kaldi-active-grammar

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#36

Post by raidicy »

mk4 wrote: post #31
raidicy wrote: post #30

Earlier quoted context omitted.

thank you I ended up signing up for the waitlist I saw the demo for the English to bash one shot learning if I could somehow use that and combination with a speech recognition software I might be able to achieve something along the lines said I'm looking for

Not to be too forward - but maybe ask then on twitter - there are a few conversations around accessibility and perhaps you could give them feedback from your needs.

thank you for the suggestion I've never really used much of Twitter but it is worth a shot . if i just reply to their thread will they see it?

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#37

Post by lunixbochs »

xenonite wrote: post #29
byteface wrote: post #18

On a mac there is a tool called 'voice control' which can trigger custom 'commands' or keyboard shortucts. You can use it to trigger shortcuts in any IDE. So if your IDE supports custom shortcuts for templating you're away.

Indeed. And I really don't get why this has been downvoted. Yes, the OP wrote he didn't like a command driven approach. But thats on windows, which comes with its own problems, see e.g., https://news.ycombinator.com/item?id=23507363 .

Voice Control is much worse than WSR at this task, not for accuracy reasons, but for API and extensibility reasons. (I went above and beyond to try to make it work, the underlying APIs crash and/or hang if you try to load in large grammars, too much custom vocab, or repeatable commands). In Voice Control you can basically only define simple commands that require pauses, and there's no way to build a system for something like spelling words or inserting specific special characters without requiring a large. pause. between. every. single. thing. you. say.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#39

Post by downerending »

Not sure exactly how bad "severe" is, but I had a lot of luck with my RSI switching to two-fingered typing for a (long) while. It's crucial to keep everything below your elbows utterly relaxed, like a pianist, sort of.

Also, I bought a keyboard tray that supported a deep negative angle, which helped me keep a very anatomical (relaxed and natural) position.

Also, figure out that mouse, somehow. Something like the above, plus switch sides frequently.

I've no idea if that could help you, but after a few years, I'm largely in remission.

I know this isn't really what you were asking, but I'm somewhat hopeful you can find relief. Good luck.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#40

Post by tluyben2 »

Probably not good for your case, but end of the summer we are going in beta launch for our product which is a visual + speech controlled programming language. It's very niche as it's a new language and IDE, from scratch, but so far it's been fun working on it.
Post reply on HN