Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Competitive Programming with AlphaCode

deepmind.com

Post reply on HN
Print view View original

121–130 of 415 posts

Re: Competitive Programming with AlphaCode

#121

Post by qualudeheart »

jonas_kgomo wrote: post #112
qualudeheart wrote: post #107

Earlier quoted context omitted.

One of the big venture capitalists predicted “prompt engineering” as a future high paid and high status position. Essentially handling large language models. Early prompt engineers will probably be drawn from “data science” communities and will be similarly high status, well but not as well paid, and require less mathematical knowledge. I’m personally expecting an “Alignment Engineer” role monitoring AI systems for u…

I like this descriptions better, considering that companies like Anthropic are working specifically on Alignment and AI Safety. Being that the team actually spun out of Deep Mind, it is interesting.

Alignment is going be a giant industry and will also include many people not originally in Stem. The humanities and “civil society” will both have their contributions to make.

It’s likely that alignment jobs won’t themselves be automated because noone will trust AI systems to align themselves.

Re: Competitive Programming with AlphaCode

#122

Post by jdlshore »

wittycardio wrote: post #6
hmate9 wrote: post #2

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but our time will be spent much more on just the design part and writing detailed documentation/specs and then we just have one of these AIs generate the code. It's the next step. Binary code Historically its always been about abstracting and writing less code to…

Solving competitive programming problems is essentially solving hard combinatorial optimization problems. Throwing a massive amount of compute and gradient descent at the problem has always been possible. If I'm not mistaken what this does is reduce the representation of the problem to a state where it can run gradient descent and then tune parameters. The real magic is in finding structurally new approaches. If anyt…

> If anything I'd say algorithms and math continue to be the core of programming.

I disagree; I think the core of programming is analyzing things people want and expressing solutions to those wants clearly, unambiguously, and in a way that is easy to change in the future. I'd say algorithms and math are a very small part of this work.

Re: Competitive Programming with AlphaCode

#123

Post by arendtio »

Permit wrote: post #73
pretendscholar wrote: post #42

I am a little bitter that it is trained on stuff that I gave away for free and will be used by a billion dollar company to make more money. I contributed the majority of that code before it was even owned by Microsoft.

Can you elaborate and give some history? What code did you contribute, and how did it end up being used by Microsoft and then DeepMind?

> We pre-train our model on selected public GitHub code and fine-tune it on our relatively small competitive programming dataset.

But since the code was 'selected' you don't know if your code was used. However, they seem to have used Python and C++, so my code is probably not part of it.

Re: Competitive Programming with AlphaCode

#124

Post by muds »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

It can be really tempting to think about research progression on a "linear" timescale but more often than not it eventually ends up following an "exponential" curve because of technical debt. And there appears to be a _lot_ of techniques used here which we don't fully understand.

I wouldn't be surprised if a specifically engineered system ten years from now wins an ICPC gold medal but I'm pretty sure that a general purpose specification -> code synthesizer that would actually threaten software engineering would require us to settle a lot of technical debts first -- especially in the area of verifying code/text generation using large language models.

Re: Competitive Programming with AlphaCode

#125

Post by baobabKoodaa »

FemmeAndroid wrote: post #103

This is extremely impressive, but I do think it’s worth noting that these two things were provided: - a very well defined problem. (One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem, not something I experience on most days.) - existing test data. This is definitely a great accomplishment, but I think those two features of competitive program…

> One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem

English versions of Codeforces problems may be well-defined but they are often very badly articulated and easy to misunderstand as a human reader. I still can't understand how they got AI to be able to generate plausible solutions from these problem statements.

Re: Competitive Programming with AlphaCode

#126

Post by tluyben2 »

udev wrote: post #101
qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

Yes, for very precise, comprehensive text descriptions of problems. It will take a far-far more advanced AI to write such descriptions for real-world problems. Writing requirements for a project is difficult work, and not for technical reasons, but for human reasons (people don't know what they want exactly, people have trouble imagining things they haven't seen yet, people are irrational, people might want something…

Yes, they have been trying to create 'sufficiently formal human readable text' to spec out projects; not detailed enough to execute by a computer but formal and precise enough so humans know exactly what they are getting. That still doesn't work at all and that is between humans. If the specs are clear enough, the act of programming is already mostly not the issue, however, they never are. I am looking forward to ML helping me writing boring code (which CoPilot already does, but again, that's not really where time/energy is spent anyway) and protect against security issues, scalability issues and all kinds of bugs (it could rewrite algo's it knows; it could recommend libraries that I should use instead of the crap I rolled myself etc).

Re: Competitive Programming with AlphaCode

#127

Post by Enginerrrd »

diehunde wrote: post #39
hmate9 wrote: post #2

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but our time will be spent much more on just the design part and writing detailed documentation/specs and then we just have one of these AIs generate the code. It's the next step. Binary code Historically its always been about abstracting and writing less code to…

My bet would be that it will never happen in a reasonable time frame. And also by that logic, writing that "documentation/spec" would just mean learning a new programming language the AI engine can parse making it as useful as a compiler. Anyone who has been writing and designing software for a while knows the cycle is way more complex than take some input and write code. Let me know when the AI engine is able to do…

I agree, from a totally different angle. Let's take something I know better as an example: Structural engineering. Structural engineering should be a "solved problem". It seems, ostensibly, relatively simple compared to a more open ended activity like "programming".(For "technical reasons", it ends up being more similar than you might think.) Still, you are ultimately dealing with the same materials, the same physics, and very similar configurations.

And yet, despite the fact that we have programs to help calculate all the things, test code-required load-combinations, even run simulations and size individual components... it turns out that, it doesn't actually save that much work, and you still need an engineer to do most of it. And not just because of regulatory requirements. It's just, that's not the hard part. The hard part is assembling the components and specifications, specifying the correct loads based on location-specific circumstances, coming up with coherent and sensible design ideas, chasing down every possible creative nook and cranny of code to make something that was originally a mistake actually work, and know when the model is just wrong for some reason and the computer isn't simulating load paths accurately.

Specifying the inputs and interpreting results is still about as much work as it was before you started with all the fancy tools. Those tools still have advantages mind you, and they do make one slightly more efficient. Substantially so in some cases, but most of the time it still comes out as a slight assist rather than a major automation.

Re: Competitive Programming with AlphaCode

#128

Post by CobrastanJorji »

qualudeheart wrote: post #115
udev wrote: post #101

Earlier quoted context omitted.

Yes, for very precise, comprehensive text descriptions of problems. It will take a far-far more advanced AI to write such descriptions for real-world problems. Writing requirements for a project is difficult work, and not for technical reasons, but for human reasons (people don't know what they want exactly, people have trouble imagining things they haven't seen yet, people are irrational, people might want something…

Fully automating software engineering won’t happen until AGI. As a good Yuddite I expect us to have bigger problems when that happens. You need an agent with a large and coherent world model, in order to understand how your programs relate to the real world, in order to solve business tasks. This isn’t something any program synthesis tech currently available can do, because none of it has a coherent world model. GPT-…

What is a "Yuddite?" I tried Googling for it and got the impression it was LessWrong forum terminology for people who believed too strongly in LessWrong, but I couldn't find many references.

Re: Competitive Programming with AlphaCode

#129

Post by pkaye »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

How long before it can write the code without plagiarizing code from online?

Re: Competitive Programming with AlphaCode

#130

Post by Jensson »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

This is in line with what other code generation AI's have accomplished.

To reach average level at codeforces you need to be able to apply a standard operation like a sort, or apply a standard math formula, as the first 1-2 problems in the easy contests are just that. It is impressive that they managed to get this result in real contests with real unaltered questions and see that it works. But generalizing this to harder problems isn't as easy, as there you need to start to device original algorithms instead of just applying standard algorithms, for such problems the model needs to understand computer science instead of just mapping language to algorithms.

Post reply on HN