Live data from Hacker News

Ask HN: Is there any work being done in speech-to-code with deep learning?

news.ycombinator.com

51–60 of 63 posts

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#51
I was hoping this was a question about applying NLP to coding tasks, but based on the answers it is about voice to text for the special use case of coders.

I am not a coder, I am a writer. I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications?

Why can't I tell my Mac: "Computer: take this collection of files and extract all the addresses of people in Indiana."

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#52

I was hoping this was a question about applying NLP to coding tasks, but based on the answers it is about voice to text for the special use case of coders. I am not a coder, I am a writer. I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications? Why can't I tell my Mac: "Computer: take this collection of files and…

> I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications?

AI writing is currently terrible. If your writing income can be replaced by AI, you're likely summarizing simple topics (e.g. unemotional, analysis-free facts of a recent event, like a change in a stock price). In that case, your job isn't safe anyway, because there are $2/hr humans in other countries that can also take your job.

Note that all software is capable of eliminating jobs, whether it's automating writing or something else.

> Why can't I tell my Mac: "Computer: take this collection of files and extract all the addresses of people in Indiana."

NLP is nowhere near being able to generally understand requests like this. Try talking to your Google Assistant and doing anything other than a simple lookup. It fails spectacularly.

Think about all the cognition in this task: first it has to understand what you're asking it. Then it has to figure out which collection of files you mean. Then it has to understand what an "address" is and what it looks like. Finally, it has to have a concept of "Indiana" and an understanding that an address can be in Indiana.

In terms of effort, it's far more efficient for you to learn the simple programming skills to do this yourself (or to use a purpose-built tool) than to create AI that does this generally, which will cost at least billions of dollars more in research time.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#53

I was hoping this was a question about applying NLP to coding tasks, but based on the answers it is about voice to text for the special use case of coders. I am not a coder, I am a writer. I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications? Why can't I tell my Mac: "Computer: take this collection of files and…

> I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications?

I think this has to do with overestimating the power of current ML techniques combined with severely underestimating the complexity of jobs like writing. This tends to result in solutions that "overreach" and end up disappointing. If we scale back a bit, we could build some really fantastic tools that work with people.

One great example of this is the movie Spiderman: Into the Spiderverse. In order to draw the comic-book lines on faces, they used a tool that would learn over time from the artists themselves. It would guess, the artist would adjust, it would learn. This is the perfect fit for ML: try to help the human, not replace them.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#54
post #14

Windows Speech Recognition is far from the best, so perhaps your trouble could be partly caused by how you had to speak in order to be understood, rather than the command style? I used to use WSR to code by voice, and it was far more laborious than my current setup. I develop kaldi-active-grammar [0]. The Kaldi engine is state of the art for command and control. Although I don't have the data and resources for traini…

It looks like Kaldi can use different backends, which I imagine have very different performance characteristics. Can you rank them from best to worst, with relative distances?

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#55
post #52

I was hoping this was a question about applying NLP to coding tasks, but based on the answers it is about voice to text for the special use case of coders. I am not a coder, I am a writer. I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications? Why can't I tell my Mac: "Computer: take this collection of files and…

> I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications? AI writing is currently terrible. If your writing income can be replaced by AI, you're likely summarizing simple topics (e.g. unemotional, analysis-free facts of a recent event, like a change in a stock price). In that case, your job isn't safe anyway, beca…

I dunno... English to sql query is probably easier than English to French. You don't actually need to understand that Indiana is a place, just recognize that it's one of those tokens that's getting stuffed into the where clause...

The tough part is really probably the implicit knowledge of what the database columns are called. Is Indiana a "state" field or a substring in an address field? Depends on the database.

This mixing of "soft" ml queries with "hard" requirements is a really interesting space, which I think will produce massive results if we can get it under control. It's similar to the problems in the AI dungeon master space... And probably tutoring, as well...

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#56
post #52

Earlier quoted context omitted.

> I wonder why all these AI people are trying to create things that will displace my means of earning a living instead of something that will create applications? AI writing is currently terrible. If your writing income can be replaced by AI, you're likely summarizing simple topics (e.g. unemotional, analysis-free facts of a recent event, like a change in a stock price). In that case, your job isn't safe anyway, beca…

I dunno... English to sql query is probably easier than English to French. You don't actually need to understand that Indiana is a place, just recognize that it's one of those tokens that's getting stuffed into the where clause... The tough part is really probably the implicit knowledge of what the database columns are called. Is Indiana a "state" field or a substring in an address field? Depends on the database. Thi…

> I dunno... English to sql query is probably easier than English to French.

You can have substantial success in transforming English to French without understanding what is said. Some big grammar rulebooks, some dictionaries, and a few terabytes of sentence-aligned parallel texts and data on statistical colocations in the languages will get you most of the way. It's far from 100% accurate but the human brain is surprisingly good at filling in the gaps with the mangled grammar and odd turns of phrase.

I'm not so sure the same is true about English to SQL query. The exact meaning of the English must be understood, in context, in order to construct the query correctly. No room for error or fuzziness. The database engine doesn't have human cognition to fall back on should the query be slightly malformed or ambiguous.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#58
post #14

Windows Speech Recognition is far from the best, so perhaps your trouble could be partly caused by how you had to speak in order to be understood, rather than the command style? I used to use WSR to code by voice, and it was far more laborious than my current setup. I develop kaldi-active-grammar [0]. The Kaldi engine is state of the art for command and control. Although I don't have the data and resources for traini…

It looks like Kaldi can use different backends, which I imagine have very different performance characteristics. Can you rank them from best to worst, with relative distances?

Just to be clear, the Dragonfly speech recognition command and control framework has multiple "backends" (speech recognition engines), including my Kaldi one. Probably the most used one currently is the Dragon Naturally Speaking backend.

The Kaldi engine, being developed primarily for research in speech recognition, can support a huge variety of "models". I think the consensus general best for most use cases (particularly for real time, low latency, streaming use) currently would be considered to be the "nnet3 chain" models, which are what my kaldi-active-grammar uses/supports.

Re: Ask HN: Is there any work being done in speech-to-code with deep learning?

#59
I really hate to see programmers/typists suffering from RSI when it is entirely preventable with the right ergonomics. Having worked on production NLP systems, I have to say I think typing will remain a more effective way of coding for many years to come ( for many reasons, but primarily because syntaxes change often and training for syntax and context is hard ). I also had RSI for many years, and finally it started to affect me playing sports and esports.

So first, I switched my mouse to my non dominant hand ( left hand for me ), as that hand already has many things to deal with. I'm also using a workstation that allows me to mount my displays at eye-level while sitting or standing. Not hunching over is ergonomics 101. Second, I switched from a standard keyboard to a split keyboard. I tried many -- Goldtouch, Kinesis Advantage2, Kinesis Freestyle -- and ultimately settled on the Ultimate Hacking Keyboard.

I could write many more paragraphs on how I customized it and why it won out, but the most important thing is that is is split and it "felt" best, once I mastered the key placements ( arrows are in different places ).

Third, I started learning VIM. Vim is really awesome but up until recently didn't have great IDE or other editor support. Now it does so there's no reason to not use it. I mostly use it for quickly jumping around files and going to line numbers.

Fourth, I'm always looking to optimize non-vim shortcuts in my editor. For example, expand-region ( now standard in VSCode ) is one of my favorite plugins.

Fifth, I'm very conscious of using my laptop for long stretches of time. Mousing on the mousepad is much more RSI inducing than using a nice gaming mouse and the UHK keyboard.

All of this to say that RSI doesn't have to be career ending. If you're doing software work and you have functioning hands and wrists you should definitely look to optimize typing before looking to speech to code. Good luck!

Post reply on HN