Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Demo of an OpenAI language model applied to code generation [video]

twitter.com

Post reply on HN
Print view View original

141–150 of 158 posts

Re: Demo of an OpenAI language model applied to code generation [video]

#141

Post by BaronSamedi »

YeGoblynQueenne wrote: post #84

So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…

I agree. While this is well done, it seems to be copying human programming techniques rather than allowing the AI to create code that it thinks is optimal. I think there is the potential to evolve efficient and secure code that is free from the constraints we impose on it due to the way our minds work. Such code may not be intelligible to us but could very well be much better than what we could write.

Re: Demo of an OpenAI language model applied to code generation [video]

#142

Post by sailingparrot »

p1esk wrote: post #127
sailingparrot wrote: post #64

I'am a bit confused, is this built by OpenAI or Microsoft? Microsoft released the paper IntelliCode Compose: Code Generation Using Transformer [1] 4 days ago and there is no attribution to anyone from OpenAI in it. Are those two entirely separate and yet exactly similar initiatives? [1]: https://arxiv.org/abs/2005.08025v1

IntelliCode Compose is built around a multi-layer generative pretrained transformer model for code (GPT-C), which is avariant of the GPT-2 GPT-2 is built by OpenAI

I am aware of this, I am referring to the video, where Sam Altman (CEO of OpenAI) is presenting the demo and saying "we have built", while Kevin Scott (CTO of MSFT) is saying that it's the first time he has seen that. So this is clearly marketed as OpenAI's work, not just saying that the model is based on their work.

Re: Demo of an OpenAI language model applied to code generation [video]

#143

Post by YeGoblynQueenne »

joshuak wrote: post #89
YeGoblynQueenne wrote: post #84

So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…

I agree completely with your expectation of the abilities of such a system. However, I think very little programming labor is employed in the construction of new algorithms or even most business logic, even a casual stroll through github reveals a staggering amount of reimplementation. I think the promise here is the ability to code in a more conceptual way with less fiddling with the finicky details.

I agree- that's why I think such a system is not capable of innovation.

In the same way, that's why I think it would be a useful tool: it promises to automate away the kind of coding that most programmers can do with eyes closed and that's the most boring and repetitive part of the job.

Like, without trying to demean it, it sounds like a great boilerplate generator.

Re: Demo of an OpenAI language model applied to code generation [video]

#144

Post by YeGoblynQueenne »

GregarianChild wrote: post #140
YeGoblynQueenne wrote: post #86

Earlier quoted context omitted.

>> Edit: Also, everyone should go try the FlashFill feature in Microsoft excel. As far as I know, it's the only example of program synthesis shipped in a consumer facing production system, and it works shockingly well. And it's not a giant language model trained on a gigantic dataset. Rather, if memory serves, it's a buch of task-specific DSLs and rules, all hand-written from scratch.

I don't know how FlashFill works in 2020, but from [1] I learn that the original implementation was a brute-force enumeration (with clever heuristics along the lines of CDCL (= conflict-driven clause learning in SAT solvers) for speeding up common cases) of a small DSL for string manipulation. This was (and still is) the state-of-the-art approach to programming-by-example program synthesis. [1] O. Polozov, S. Gulwani…

That's a nice, formal way of putting it, thank you :)

(Sorry I really should have refreshed my memory on Gulwani et al. I think I've even linked the paper on an HN comment before.)

Oh, btw, I doubt they're doing this with a language model nowadays. Unless FlashFill has suddendly started filling cells for email addresses with haikus etc...

Re: Demo of an OpenAI language model applied to code generation [video]

#145

Post by yazr »

gwern wrote: post #92
YeGoblynQueenne wrote: post #84

So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…

I think you are underselling the potential of a model which deeply understand programming. Imagine combining such a model with something like AutoML-Zero: https://arxiv.org/abs/2003.03384 It may not be 'creative', but used as tab-completion, it's not being rewarded or incentivized or used in any way which would expose its abilities towards creating a new sort algorithm.

What do you think is a more productive path leading to "AutoCode" ?!

A. Add external definitions or reward formalism to make the code-space easier to search?

OR

B. Keep adding code trees, execution traces, comments, memory dumps and learn from those?

My own instinct is that AlphaZero was a lot more convincing than AlphaStar, so lots of (A) is definitely needed

Re: Demo of an OpenAI language model applied to code generation [video]

#146

Post by mring33621 »

ttul wrote: post #76
detay wrote: post #74

Earlier quoted context omitted.

that would be supervised ai training, until 3) programmers become obsolete.

I don't think programmers will become obsolete. They will just waste less time on boilerplate and reading Stackoverflow articles to figure out how to do XYZ. Why not have an AI do that for you so that you can focus on the creative stuff? Programming tools help us work more productively, which leads to larger and more complex systems in less time - a win for everyone.

I agree. I think this would make programming more fun and more productive.

Re: Demo of an OpenAI language model applied to code generation [video]

#147

Post by mooseburger »

detay wrote: post #74
mring33621 wrote: post #22

Amazing! So the developer's role will shift to: 1) writing good enough descriptions of the code to be generated by the AI model 2) fixing any little issues in the generated code

that would be supervised ai training, until 3) programmers become obsolete.

Once programmers become obsolete, it's likely a matter of days until the human race becomes obsolete.

Re: Demo of an OpenAI language model applied to code generation [video]

#148

Post by Avi-D-coder »

GPT2 is known to be unable to track and bind variables, scaling purely associative models beyond the trivial examples is going to be difficult or more likely impossible.

This will end up being a better tabnine. Models like GPT2 are still just approximating intelligence, they are not rationally cognizing.

Re: Demo of an OpenAI language model applied to code generation [video]

#149

Post by gradys »

Avi-D-coder wrote: post #148

GPT2 is known to be unable to track and bind variables, scaling purely associative models beyond the trivial examples is going to be difficult or more likely impossible. This will end up being a better tabnine. Models like GPT2 are still just approximating intelligence, they are not rationally cognizing.

Curious what analysis you're referencing here.

While I don't doubt people have shown that various transformer models have certain limitations, I'm pretty bullish on transformer models in general.

Here's a post exploring the application of transformers to symbolic mathematics for instance: https://medium.com/analytics-vidhya/solving-differential-equ...

Re: Demo of an OpenAI language model applied to code generation [video]

#150

Post by gradys »

YeGoblynQueenne wrote: post #84

So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…

I'd put it differently. This is going to take your job, just like an assembly programmer from the 70s might consider Python to have basically taken their job. In software, the job is constantly eating itself and transforming.

It's part of the job to continually incorporate new capabilities and lever yourself up.

Post reply on HN