Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Competitive Programming with AlphaCode

deepmind.com

Post reply on HN
Print view View original

311–320 of 415 posts

Re: Competitive Programming with AlphaCode

#311

Post by ww520 »

qualudeheart wrote: post #90

Calling it now: If current language models can solve competitive programming at an average human level, we’re only a decade or less off from competitive programming being as solved as Go or Chess. Deepmind or openAI will do it. If not them, it will be a Chinese research group on par with them. I’ll be considering a new career. It will still be in computer science but it won’t be writing a lot of code. There’ll be sev…

Didn’t we all (collectively) have this discussion the last time someone put the math functions in a library and rendered math calculation programmers obsolete?

Re: Competitive Programming with AlphaCode

#312

Post by xmprt »

orangecat wrote: post #132
xmprt wrote: post #66

Earlier quoted context omitted.

I don't think I agree. Most people spend more time reading than writing code so programming languages should be optimized to be easier to read whereas tooling should be made to simplify writing code. New syntax or macros sounds like it would make the language harder to read. I agree that an advanced machine learning model for generating boilerplate code isn't the right approach but I also don't think we should extend…

New syntax or macros sounds like it would make the language harder to read. Often the opposite is true. For example Java records are far easier to read and understand than the pages of boilerplate that they replace.

That sounds like an issue with how Java was designed. There are plenty of languages that solve Java's boilerplate problems without adding new syntax for records.

Re: Competitive Programming with AlphaCode

#313

Post by gavagai691 »

dwohnitmok wrote: post #285
gavagai691 wrote: post #276

Earlier quoted context omitted.

Two possible reasons. 1. First, though I am not sure of this (i.e. this should be verified), I heard that the team working on AlphaStar initially tried to create a Starcraft AI entirely through "self-play," but this was not successful. (Intuitively, in a real-time game, there are too many bad options too early on that even with a LOT of time to learn, if your approach is too "random" you will quickly enter an unwinna…

> but he could not use any custom hotkeys. IIRC you could and Serral did set his own custom keybindings on the machine. The main difference was different keyboard and mouse.

I looked into this again and the hotkey situation seems more unclear than I suggested. You could not log into your Battle.net account, so it would have been somewhat time consuming to change all of your settings manually. If I had to guess, I might wager that Serral changed some of the more important ones manually but not the others, but this is just conjecture and maybe he changed all of them. I don't know if anyone but Serral would know this, however.

In any case, Serral said this, which you can take as you will:

https://twitter.com/ENCE_Serral/status/1192023800961019904

"It was okay, I doubt i would lose too many games with a proper setup. I think the 6.3-6.4 mmr is pretty accurate, so not bad at all but nothing special at the same time."

On the one hand, surely it doesn't seem surprising that the player who lost, the human, would say the above, and so one may be skeptical of how unbiased Serral's assessment is. On the other hand, I would say that Serral is among the more frank and level-headed players I've seen in the various videogames I've followed, so I wouldn't be too hasty to write off his assessment for this reason.

Re: Competitive Programming with AlphaCode

#314

Post by errcorrectcode »

And this is how we reach the technological singularity and how programmers become as equivalently out-of-demand as piano tuners: self-programming systems.

AI will eat any and all knowledge work because there's very little special a human can do that a machine won't be able to do eventually, and much faster and better. It won't be tomorrow, but the sands are inevitably shifting this way.

Re: Competitive Programming with AlphaCode

#315

Post by gavagai691 »

fuy wrote: post #303
gavagai691 wrote: post #276

Earlier quoted context omitted.

Two possible reasons. 1. First, though I am not sure of this (i.e. this should be verified), I heard that the team working on AlphaStar initially tried to create a Starcraft AI entirely through "self-play," but this was not successful. (Intuitively, in a real-time game, there are too many bad options too early on that even with a LOT of time to learn, if your approach is too "random" you will quickly enter an unwinna…

"It seemed like they cut the project off at a strange point, before AlphaStar was clearly better than humans. I feel that if they had continued they could have gotten to that point" What if that's why they cut it off..

Apologies, I don't quite follow your reasoning.

Re: Competitive Programming with AlphaCode

#316

Post by d0mine »

falcor84 wrote: post #192
d0mine wrote: post #168

It reminds me that median reputation on StackOverflow is 1. All AlphaSO would have to do is to register to receive median reputation on SO ;) (kidding aside AlphaCode sounds like magic) Inventing relational DBs hasn't replaced programmers, we just write custom DB engines less often. Inventing electronic spreadsheets hasn't deprecated programmers, it just means that we don't need programmers for corresponding tasks (w…

>AI won't replace programmers until it grows to replace the humanity as a whole. Yes, but after seeing this progress in the former, my time estimate of time remaining until the latter had just significantly shortened.

Given close to zero chances of a safe AI, I'm optimistic that AI is a much tougher problem and we are not significantly closer to the solution than e.g., in 60s when computer vision was a summer project.

There is a progress in certain domains (such as image recognition) but (outside specialized tasks) gigantic language models look like no more than impressive BS generators.

Re: Competitive Programming with AlphaCode

#317

Post by iab »

doctor_eval wrote: post #267

I sometimes read these and wonder if I need to retrain. At my age, I’ll struggle to get a job at a similar level in a new industry. And then I remember that the thing I bring to the table is the ability to turn domain knowledge into code. Being able to do competitive coding challenges is impressive, but a very large segment of software engineering is about eliciting what the squishy humans in management actually want…

I think a good yardstick for this is something that is generative; so, for instance, can the system generate a good programming challenge question? This is still a no.

Re: Competitive Programming with AlphaCode

#318

Post by YeGoblynQueenne »

Veedrac wrote: post #300
YeGoblynQueenne wrote: post #297

Earlier quoted context omitted.

Yes, apparently they couldn't use their full approach because "of missing information in the dataset". That points to a further limitation of the approach: it works for Codeforces problems but not for APPS problems (so it's very purpose-specific). Btw, APPS is not much of a benchmark. It evaluates code generation according to how close it resembles code written by humans. That's standard fare for text generation benc…

> Btw, APPS is not much of a benchmark. It evaluates code generation according to how close it resembles code written by humans. No, the metric used in this paper was the percentage of questions it could solve against the hidden tests. > That points to a further limitation of the approach: it works for Codeforces problems but not for APPS problems (so it's very purpose-specific). This does not matter typically since…

>> No, the metric used in this paper was the percentage of questions it could solve against the hidden tests.

Right, that's my mistake. The APPS dataset has natural language specifications and test cases for evaluation. It actually includes Codeforces problems.

The excuse quoted in the second part of your comment is an excuse. If a large language model can complete a code generation task, that's because it's seen an example of the code it's asked to generate before. Any claims to the contrary need very strong evidence to support them and there's typically no such thing in papers like the AlphaCode one.

Re: Competitive Programming with AlphaCode

#319

Post by errcorrectcode »

deanmen wrote: post #214
37ef_ced3 wrote: post #36

Earlier quoted context omitted.

This is a toy problem to illustrate that CoPilot cannot write code that requires mathematical reasoning. It regurgitates solutions from the training set, via a mixed internal reresentation.

unsigned int swapbits(unsigned int a) { bool bit6 = a & (1

Gross and not portable C99.

    #define B6 (1
Here's some BFP:

    unsigned swapbits(unsigned a) {
       unsigned flip = (a & B6 == a & B17);
       return (a ^ ((flip
int and double are C's implicit lingua francas for underspecified literals and implicit type conversions. Throwing int everywhere is redundant like "ATM machine."

Re: Competitive Programming with AlphaCode

#320

Post by errcorrectcode »

ensan wrote: post #282

Wake me up when an AI creates an operating system on the same level of functionality as early-years Linux.

That will happen faster than you can conceive because you won't be aware of the progress until it is announced.

And, have you tried polling? I hear it keeps the CPU warm in winter. Interrupts are so ... this just in, Nike's stock jump 3% ... Where was I? Did I save my task context properly? Did I reenable interrupts?

Post reply on HN