21–30 of 63 posts
I came across Serenade ( https://serenade.ai/ ) recently. It's still beta but I was very impressed. In the past I've used vocola, and a few other open-source options. Serenade felt much more natural and powerful. The founders are also super hands-on and genuinely seem to care about the problem.
There are a few good OSS offline deep speech libraries including Mozilla DeepSpeech [3], but their resource footprint is too high. We settled on the currently less mature vosk [4], which is based on Kaldi [5] (a more popular deep speech pipeline), and includes a number of low-footprint, pretrained language models for real-time streaming inference. Research has shown how to deploy efficient deep speech models on CPUs [6], so we're hoping those gains will translate to faster performance on commodity laptops soon. You can follow this issue [7] for updates on our progress. Contributions are welcome!
[1]: https://github.com/OpenASR/idear/
[2]: https://cmusphinx.github.io/
[3]: https://github.com/mozilla/DeepSpeech
[4]: https://github.com/alphacep/vosk-api
[5]: https://github.com/kaldi-asr/kaldi
[6]: https://ai.facebook.com/blog/a-highly-efficient-real-time-te...
I can’t help you with actually converting speech to code but it occurs to me this would be a benefit to everyone. Speaking the words that are represented by the code we write would require a much deeper understanding of what we are doing and why. Food for thought for sure. Good luck.
I have found this research paper[0] that describes using a machine translation with a CNN and a r n n. it seems one of the main problems that come up with is a lack of training data so to progress this further it seems like we would need a big corpus of parallel text between natural English describing what the code does in the actual source code. I only have some novice experience with training machine learning models through transfer learning with fast AI but I have found a repository of tutorials[1] that teach you how to do machine translation between two languages and it seems maybe this approach might be applicable. There is a small corpus of text[2] with python code with annotations but and the paper itself at States. It probably needs more to get a higher than a 74% accuracy.
the only thing I can really think is to scrape GitHub make a website where you can get people to crowdsource annotations for code samples. I'm not sure on how to deal with how much and how little code would be for a sampling. I eat how much of the code would you need for a good training sample versus just using isolated functions. Perhaps that something like this might be used in correspondence with GPT.
[0]https://www.groundai.com/project/machine-translation-from-na... [1]https://github.com/bentrevett/pytorch-seq2seq [2]https://ahclab.naist.jp/pseudogen/
I can’t help you with actually converting speech to code but it occurs to me this would be a benefit to everyone. Speaking the words that are represented by the code we write would require a much deeper understanding of what we are doing and why. Food for thought for sure. Good luck.
There's been some pretty cool work in this area recently: https://www.youtube.com/watch?v=fZSFNUT6iY8
To me it seems like learning how to talk to Alexa or Cortana or "Google" is a limitation or regression for humans. This shows that it could actually be beneficial.
Thanks for this philosophical rabbit hole just in time for a weekend.
Are you sure you have RSI and not TMS (tension myositis syndrome)? It's a condition that causes real physical symptoms (of which wrist pain is a common one) that are not due to pathological or structural abnormalities. Rather, the symptoms are caused by stress and repressed emotions. Check out this success forum of people who have healed from all kinds of chronic pain symptoms by dealing with stress and changing thei…
Rarely does the RSI flare up when I'm doing something I enjoy. Stress is definitely a component.
I've been working on a similar use case at work (going from discoursive speech to cli-like commands, using a semi-rigid language), and I didn't find any off-the-shelf purely ML-based solution that would work for us. In my experience, I've found any services claiming to do deep learning produced far worse results than what we could get with simple approaches. That is, when faced with non-grammatical sentences (or rath…
You've see seen openai's new english -> bash demo right? That said, Silvius is more of demo than a product, the IMO best voice programming options right now are (in alphabetical order): - Caster/dragonfly (fully open-source if you use daanzu's Kaldi engine, which is way better than Silvius afaik, I think even the creator of silvius uses dragonfly with dragon instead of using silvius) - Serenade (fully commercial, I h…
Not yet, but will do, thanks!
However, I'd still be hesitant to build a product on top of that: Does voice to bash help us if we now want to do, say, voice to python? At least we'd need to re-train the system with completely new data, and even if we use transfer learning to our advantage, it's not an easy task. There's also no guarantees that the chosen neural network architecture that works for bash, will work the same for any programming language (think of a radically different syntax, like Lisp for example).
The training must also be re-done for any variation in the input format to some extent. i.e., accent, expected background noise levels, and of course (human speaker) language.
ML has its use case, but I typically see these nice demos as that, demos. When you have to build a real product and solve user problems, you can't rely on a black box doing what you want.
Earlier quoted context omitted.
You've see seen openai's new english -> bash demo right? That said, Silvius is more of demo than a product, the IMO best voice programming options right now are (in alphabetical order): - Caster/dragonfly (fully open-source if you use daanzu's Kaldi engine, which is way better than Silvius afaik, I think even the creator of silvius uses dragonfly with dragon instead of using silvius) - Serenade (fully commercial, I h…
> You've see seen openai's new english -> bash demo right? Not yet, but will do, thanks! However, I'd still be hesitant to build a product on top of that: Does voice to bash help us if we now want to do, say, voice to python? At least we'd need to re-train the system with completely new data, and even if we use transfer learning to our advantage, it's not an easy task. There's also no guarantees that the chosen neura…
Also they do have a related Python demo: https://news.ycombinator.com/item?id=23507145
Speech is a completely different stack to this, but honestly (english) speech is much more of a solved problem here than general knowledge.
Earlier quoted context omitted.
There's been some pretty cool work in this area recently: https://www.youtube.com/watch?v=fZSFNUT6iY8
Wow. This is exactly what I imagined but it already exists. This is a great illustration of how we as humans could use our abilities to disambiguate to collaborate with a computer and write code. Very impressive stuff. To me it seems like learning how to talk to Alexa or Cortana or "Google" is a limitation or regression for humans. This shows that it could actually be beneficial. Thanks for this philosophical rabbit…
On a mac there is a tool called 'voice control' which can trigger custom 'commands' or keyboard shortucts. You can use it to trigger shortcuts in any IDE. So if your IDE supports custom shortcuts for templating you're away.
Not coding - but openai's API in beta has a speech to bash function https://openai.com/blog/openai-api/