Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Competitive Programming with AlphaCode

deepmind.com

Post reply on HN
Print view View original

131–140 of 415 posts

Re: Competitive Programming with AlphaCode

#131

Post by zerr »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

The thing is, Competitive Programming (CP) is a completely different discipline/subject with its own trivia knowledge and tricks. CP uses Computer Science the same way as e.g. Biology uses Mathematics. It has very little in common with a real world software development.

Re: Competitive Programming with AlphaCode

#132

Post by orangecat »

xmprt wrote: post #66
chongli wrote: post #62

Earlier quoted context omitted.

repetitive code like this.foo = foo; this.bar = bar etc... This sort of boilerplate code is best solved by the programming language. Either via better built-in syntax or macros. Using an advanced machine learning model to generate this code is both error-prone and a big source of noise and code bloat. This is not an issue that will go away with better tooling; it will only get worse.

I don't think I agree. Most people spend more time reading than writing code so programming languages should be optimized to be easier to read whereas tooling should be made to simplify writing code. New syntax or macros sounds like it would make the language harder to read. I agree that an advanced machine learning model for generating boilerplate code isn't the right approach but I also don't think we should extend…

New syntax or macros sounds like it would make the language harder to read.

Often the opposite is true. For example Java records are far easier to read and understand than the pages of boilerplate that they replace.

Re: Competitive Programming with AlphaCode

#133

Post by buscoquadnary »

ohwellhere wrote: post #117
jakub_g wrote: post #109

Earlier quoted context omitted.

100% agree. Someone (who?) had to take time and write the detailed requirements. In real jobs you rarely get good tickets with well defined expectations; it's one of most important developer's jobs to transform fuzzy requirement into a good ticket. (Side note: I find that many people skip this step, and go straight from fuzzy-requirement-only-discussed-on-zoom-with-Bob to code; open a pull request without much contex…

Is the next step in the evolution of programming having the programmer become the specifier? Fuzzy business requirements -> programmer specifies and writes tests -> AI codes

That's all we've ever been since we invented software.

First we specified the exact flow of the bits with punch cards.

Then we got assembly and we specified the machine instructions.

Then we got higher level languages and we specified how the memory was to be managed and what data to store where.

Now we have object oriented languages that allow us to work with domain models, and functional languages that allow us to work data structures and algorithms.

The next level may be writing business rules, and specifying how services talk to each other, who knows, but it will be no different than it is now just a higher level.

Re: Competitive Programming with AlphaCode

#134

Post by e4e78a06 »

FemmeAndroid wrote: post #103

This is extremely impressive, but I do think it’s worth noting that these two things were provided: - a very well defined problem. (One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem, not something I experience on most days.) - existing test data. This is definitely a great accomplishment, but I think those two features of competitive program…

I don't think it's quite as impressive as you make it out to be. Median performance in a Codeforces programming competition is solving the easiest 1-2 problems out of 5-6 problems. Like all things programming the top 1% is much, much better than the median.

There's also the open problem of verifying correctness in solutions and providing some sort of flag when the model is not confident in its correctness. I give it another 5 years in the optimistic case before AlphaCode can reliably compete at the top 1% level.

Re: Competitive Programming with AlphaCode

#135

Post by mcast »

The year is 2025, Google et al. are now conducting technical on-site interviews purely with AI tools and no human bias behind the camera (aside from GPT-3's quirky emotions). The interview starts with a LC hard, you're given 20 minutes -- good luck!

Re: Competitive Programming with AlphaCode

#136

Post by ctoth »

diehunde wrote: post #39
hmate9 wrote: post #2

Between this and OpenAI's Github Copilot "programming" will slowly start dying probably. What I mean by that is that sure, you have to learn how to program, but our time will be spent much more on just the design part and writing detailed documentation/specs and then we just have one of these AIs generate the code. It's the next step. Binary code Historically its always been about abstracting and writing less code to…

My bet would be that it will never happen in a reasonable time frame. And also by that logic, writing that "documentation/spec" would just mean learning a new programming language the AI engine can parse making it as useful as a compiler. Anyone who has been writing and designing software for a while knows the cycle is way more complex than take some input and write code. Let me know when the AI engine is able to do…

You ever notice how the "let me know when" part of this keeps changing? Let me know when computers can ... play Go/understand a sentence/compose music/write a program/ ...

But surely they'll never be able to do this new reference class you have just now come up with, right?

Re: Competitive Programming with AlphaCode

#137

Post by nikkwong »

CobrastanJorji wrote: post #128
qualudeheart wrote: post #115

Earlier quoted context omitted.

Fully automating software engineering won’t happen until AGI. As a good Yuddite I expect us to have bigger problems when that happens. You need an agent with a large and coherent world model, in order to understand how your programs relate to the real world, in order to solve business tasks. This isn’t something any program synthesis tech currently available can do, because none of it has a coherent world model. GPT-…

What is a "Yuddite?" I tried Googling for it and got the impression it was LessWrong forum terminology for people who believed too strongly in LessWrong, but I couldn't find many references.

I believe he's referring to "luddites" -- a group of people who resisted technological innovation during the industrial revolution.

Re: Competitive Programming with AlphaCode

#138

Post by Jensson »

e4e78a06 wrote: post #134
FemmeAndroid wrote: post #103

This is extremely impressive, but I do think it’s worth noting that these two things were provided: - a very well defined problem. (One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem, not something I experience on most days.) - existing test data. This is definitely a great accomplishment, but I think those two features of competitive program…

I don't think it's quite as impressive as you make it out to be. Median performance in a Codeforces programming competition is solving the easiest 1-2 problems out of 5-6 problems. Like all things programming the top 1% is much, much better than the median. There's also the open problem of verifying correctness in solutions and providing some sort of flag when the model is not confident in its correctness. I give it…

Top 1% competitive programming level means that it can start solving research problems, problem difficulty and creativity needed for problems goes up exponentially for harder problems and programming contests have lead to research papers before. It would be cool if we got there in 5 years but I doubt it. But if we got there it would revolutionize so many things in society.

Re: Competitive Programming with AlphaCode

#139

Post by qualudeheart »

buscoquadnary wrote: post #119
qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

The problem is this view continues to view software engineers as people that write code, that's not what my job is, it is figuring out how to solve a business problem using technology, and getting people on board with that solution and updating and refining it. This viewpoint seems to me to be very similar to the idea of 3rd generation languages replacing developers because programming will be so easy, it isn't about…

I actually agree with you on that. I had another comment further down the thread where I said that software engineering can’t be fully automated by anything short of artificial general intelligence.

As humans we have a coherent world model that current AI systems are nowhere near close to having.

That coherent world model is a necessary precondition for both understanding a business goal and implementing a program to solve it. AlphaCode can do the second part but not the first.

AlphaCode doesn’t have that world model and even if it did it still wouldn’t autonomously act on it, just follow orders from humans.

Competitive programming is going to be solved much earlier than programming in a business context will, because it’s completely independent of business requirements. It’s at most half as hard of a problem .

Re: Competitive Programming with AlphaCode

#140

Post by abecedarius »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

Three months ago in the Copilot thread I was saying

> in 5 years will there be an AI that's better than 90% of unassisted working programmers at solving new leetcode-type coding interview questions posed in natural language?

and getting pooh-poohed. https://news.ycombinator.com/item?id=29020401 (And writing that, I felt nervous that it might not be aggressive enough.)

There's this general bias in discussions of AI these days, that people forget that the advance they're pooh-poohing was dismissed in the same way as probably way off in the indefinite future, surprisingly recently.

Post reply on HN