Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Competitive Programming with AlphaCode

deepmind.com

Post reply on HN
Print view View original

31–40 of 415 posts

Re: Competitive Programming with AlphaCode

#31

Post by ipnon »

msoad wrote: post #19

This seems to have a narrower scope than GitHub Copilot. It generates more lines of code to a more holistic problem vs. GitHub Copilot that works as a "more advanced autocomplete" in code editors. Sure Copilot can synthesize full functions and classes but for me, it's the most useful when it suggests another test case's title or writes repetitive code like this.foo = foo; this.bar = bar etc... Having used Copilot I c…

The big question seems to be whether par with professional programmers is a matter of increasing training set and flop size, or whether different model or multi-model architectures are required.

It does look like we've entered an era where programmers who don't use AI assistants will be disadvantaged, and that this era has an expiration date.

Re: Competitive Programming with AlphaCode

#32

Post by dskloet »

37ef_ced3 wrote: post #5

The example problem (essentially, is T a subsequence of S with deletions of size N) is a classic problem with no doubt dozens of implementations in AlphaCode's training set. And yet, what a garbage solution it produces. To illustrate the difference between intelligence and regurgitation, someone tell me what CoPilot generates for this: // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed inte…

There's really no need for an 11 in the code. I'd say that makes the code worse, not better.

Re: Competitive Programming with AlphaCode

#33

Post by paxys »

hackinthebochs wrote: post #20
Inufu wrote: post #18

Earlier quoted context omitted.

I agree, I expect programmers will just move up the levels of abstraction. I enjoyed this recent blog post on the topic: https://eli.thegreenplace.net/2022/asimov-programming-and-th...

The "problem" is that as you move up the levels of abstraction, you need fewer people to do the same amount of work. Unless the complexity of the work scales as well. I've always felt that programmers would be the first class of knowledge workers to be put out of work by automation. This may be the beginning of the end for the programming gravy train.

> as you move up the levels of abstraction, you need fewer people to do the same amount of work

Yes, but the total amount of work (and surrounding complexity) also increases with it. Just look at the evolution of the software industry over the last few decades.

Re: Competitive Programming with AlphaCode

#34

Post by 37ef_ced3 »

altcognito wrote: post #16
37ef_ced3 wrote: post #5

The example problem (essentially, is T a subsequence of S with deletions of size N) is a classic problem with no doubt dozens of implementations in AlphaCode's training set. And yet, what a garbage solution it produces. To illustrate the difference between intelligence and regurgitation, someone tell me what CoPilot generates for this: // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed inte…

Would we be able to generate unit tests? Strikes me that this would be important to verify given that we didn't even "write" the code. At some point we might not even be looking at the generated code? I almost guarantee that's what is going to happen eventually.

You can see it happening already.

Solutions are posted, and they're wrong.

But the CoPilot user can't see the code is wrong.

Re: Competitive Programming with AlphaCode

#35

Post by visarga »

hackinthebochs wrote: post #20
Inufu wrote: post #18

Earlier quoted context omitted.

I agree, I expect programmers will just move up the levels of abstraction. I enjoyed this recent blog post on the topic: https://eli.thegreenplace.net/2022/asimov-programming-and-th...

The "problem" is that as you move up the levels of abstraction, you need fewer people to do the same amount of work. Unless the complexity of the work scales as well. I've always felt that programmers would be the first class of knowledge workers to be put out of work by automation. This may be the beginning of the end for the programming gravy train.

> The "problem" is that as you move up the levels of abstraction, you need fewer people to do the same amount of work.

This will lower the entry barrier to developing software so more people will go into the field. Before you needed to know a programming language, now you will just have a dialogue with a language model.

> I've always felt that programmers would be the first class of knowledge workers to be put out of work by automation.

We've been automating our work for 70 years, and look how many programmers are employed now. The more we automate, the more capable our field becomes and more applications pop up.

Re: Competitive Programming with AlphaCode

#36

Post by 37ef_ced3 »

dskloet wrote: post #32
37ef_ced3 wrote: post #5

The example problem (essentially, is T a subsequence of S with deletions of size N) is a classic problem with no doubt dozens of implementations in AlphaCode's training set. And yet, what a garbage solution it produces. To illustrate the difference between intelligence and regurgitation, someone tell me what CoPilot generates for this: // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed inte…

There's really no need for an 11 in the code. I'd say that makes the code worse, not better.

This is a toy problem to illustrate that CoPilot cannot write code that requires mathematical reasoning. It regurgitates solutions from the training set, via a mixed internal reresentation.

Re: Competitive Programming with AlphaCode

#38

Post by sharemywin »

msoad wrote: post #19

This seems to have a narrower scope than GitHub Copilot. It generates more lines of code to a more holistic problem vs. GitHub Copilot that works as a "more advanced autocomplete" in code editors. Sure Copilot can synthesize full functions and classes but for me, it's the most useful when it suggests another test case's title or writes repetitive code like this.foo = foo; this.bar = bar etc... Having used Copilot I c…

To me it's not about it's current capabilities. It's the trajectory. This tech wasn't even a thing 2 years ago. There's billions being poured into it and every time someone uses these tools there's more free training data.

Re: Competitive Programming with AlphaCode

#39

Post by diehunde »

hmate9 wrote: post #2

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but our time will be spent much more on just the design part and writing detailed documentation/specs and then we just have one of these AIs generate the code. It's the next step. Binary code Historically its always been about abstracting and writing less code to…

My bet would be that it will never happen in a reasonable time frame. And also by that logic, writing that "documentation/spec" would just mean learning a new programming language the AI engine can parse making it as useful as a compiler. Anyone who has been writing and designing software for a while knows the cycle is way more complex than take some input and write code.

Let me know when the AI engine is able to do complex refactoring or adding features that keeps backwards compatibility, find a bug in a giant codebase by debugging a test case or write code that's performant but also maintainable.

Post reply on HN