Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Competitive Programming with AlphaCode

deepmind.com

Post reply on HN
Print view View original

341–350 of 415 posts

Re: Competitive Programming with AlphaCode

#341

Post by atleta »

GuB-42 wrote: post #304
atleta wrote: post #286

Earlier quoted context omitted.

The thing is that we don't know. What I also have been seeing for a while (like for at least for a decade) that whatever profession seemed to be in danger, whichever profession came out on top on (guess) lists like "these will be replaced by AI soon", each and every one of them thought that it can't happen to them and they all had (and continue to have) explanations, usually involving how that jobs needs human ingenu…

I think programmers are relatively "safe" from AI for the simple reason they are the ones who talk to AI. The job of programmers is to have machines do stuff so that humans don't have to, and of course, they do it for themselves too. Scripts, libraries, compilers, they are just tools to avoid flipping bits by hand. If something like copilot is not embraced by all programmers, it is that it is often less than helpful,…

What I was trying to convey is that I'm not sure at all that you'll need a programmer (i.e. someone who has the mindset and the skills of a person we call today as such) to talk to the AI. Because the AI may just be able to understand a sloppy description that the average user (or the average product owner) is able to communicate. And when/if not then it will be able to either ask clarification questions (like "what do you mean by account?") or just generate something and then let the user figure out if it's doing the right thing for them. If not, they can ask for changes or explain what they think was misunderstood.

And my (weak) conjecture is that we may not need an AGI/human level AI for this. In which case we might still want to have some software to be written. But you're right, I'm also not sure that there will be a point where we still want software but have very intelligent machines. And while saying that programmer will be the last technical job doesn't sound like a strong claim, I'd say say it would probably be teachers :)

> The job will evolve, but there will always be someone who tells the computer what to do.

Which may very well be the users, if the machine is able to follow a conversation. Now the thing that may be the showstopper for now might exactly be this: that the machine should be able to hold a context for long enough (over multiple iterations of back and forth communication). As far as my limited knowledge goes, this is something that they have not yet figured out.

The "our kind will always be needed" is exactly the fallacy I was talking about and the one that the practitioners of every intellectual professions seem to have. They think they will be needed to interface between the machine (whether it's a legal or a medical system) and the client. Because they assume that the machine will not be able to communicate only to process the existing knowledge base.

But again, the whole field evolves through surprising leaps. Yep, Copilot is not insanely useful, but already amusing/frightening enough. It seems to pick up context from all over the code base. Sometimes it goes totally wrong, and generates gibberish (I mean generate non existent identifiers that make sense as English expressions but ones that don't exist anywhere in the code). But quite a few times it picks up the intent (the pattern/thought pattern) even if it is spread out over a file (or several ones).

Re: Competitive Programming with AlphaCode

#342

Post by elisbce »

GuB-42 wrote: post #304
atleta wrote: post #286

Earlier quoted context omitted.

The thing is that we don't know. What I also have been seeing for a while (like for at least for a decade) that whatever profession seemed to be in danger, whichever profession came out on top on (guess) lists like "these will be replaced by AI soon", each and every one of them thought that it can't happen to them and they all had (and continue to have) explanations, usually involving how that jobs needs human ingenu…

I think programmers are relatively "safe" from AI for the simple reason they are the ones who talk to AI. The job of programmers is to have machines do stuff so that humans don't have to, and of course, they do it for themselves too. Scripts, libraries, compilers, they are just tools to avoid flipping bits by hand. If something like copilot is not embraced by all programmers, it is that it is often less than helpful,…

Your logic is completely flawed. If there is a super AI with real intelligence that understands problems and codes it up for you, why wouldn't it be possible to go one step further and solve problems on its own? Why do you think that a human programmer has to feed a problem statement to the AI for it to work?

Re: Competitive Programming with AlphaCode

#343

Post by deanmen »

errcorrectcode wrote: post #319
deanmen wrote: post #214

Earlier quoted context omitted.

unsigned int swapbits(unsigned int a) { bool bit6 = a & (1

Gross and not portable C99. #define B6 (1 Here's some BFP: unsigned swapbits(unsigned a) { unsigned flip = (a & B6 == a & B17); return (a ^ ((flip int and double are C's implicit lingua francas for underspecified literals and implicit type conversions. Throwing int everywhere is redundant like "ATM machine."

The definition of flip requires parenthesis (a & B6) == (a & B17) as == has higher precedence than and. int is required in C++ but not in C as you said.

Re: Competitive Programming with AlphaCode

#344

Post by GuB-42 »

elisbce wrote: post #342
GuB-42 wrote: post #304

Earlier quoted context omitted.

I think programmers are relatively "safe" from AI for the simple reason they are the ones who talk to AI. The job of programmers is to have machines do stuff so that humans don't have to, and of course, they do it for themselves too. Scripts, libraries, compilers, they are just tools to avoid flipping bits by hand. If something like copilot is not embraced by all programmers, it is that it is often less than helpful,…

Your logic is completely flawed. If there is a super AI with real intelligence that understands problems and codes it up for you, why wouldn't it be possible to go one step further and solve problems on its own? Why do you think that a human programmer has to feed a problem statement to the AI for it to work?

It is what I meant by "the singularity". AIs that are so intelligent that they don't need humans, including when it comes to building better AIs. The idea is that they get in a runaway self-improving cycle and what they do after that and the place of humanity is left to the imagination.

I don't believe in the singularity, but if we get to the point where AIs don't need human programmers anymore, things are going to get... interesting.

Re: Competitive Programming with AlphaCode

#345

Post by chongli »

xmprt wrote: post #312
orangecat wrote: post #132

Earlier quoted context omitted.

New syntax or macros sounds like it would make the language harder to read. Often the opposite is true. For example Java records are far easier to read and understand than the pages of boilerplate that they replace.

That sounds like an issue with how Java was designed. There are plenty of languages that solve Java's boilerplate problems without adding new syntax for records.

If you’ll review my original comment, I never said new syntax. I said better syntax. If your language design leads to a lot of boilerplate in idiomatic use then it needs to be better. Adding new syntax is just putting a bandaid on the problem.

Re: Competitive Programming with AlphaCode

#346

Post by rfoo »

Veedrac wrote: post #292
Jensson wrote: post #190

Earlier quoted context omitted.

Likely the python runtime has a strange string implementation for cases like this, just like javascript strings.

It does not. Really the strings just never get long enough that O(n²) would be catastrophic; the maximum possible length is 2e5.

2e5 is enough for making a naive O(n^2) solution to get TLE.

This is likely due to the fact that in AlphaCode's solution the "inner O(n) loop" is actually a memmove(), which is optimized to be insanely fast.

Re: Competitive Programming with AlphaCode

#347

Post by MattRix »

briga wrote: post #280
FiberBundle wrote: post #217

It never ceases to amaze me what you can do with these transformer models. They created millions of potential solutions for each problem, used the provided examples for the problems to filter out 99% of incorrect solutions and then applied some more heuristics and the 10 available submissions to try to find a solution. All these approaches just seem like brute-force approaches: Let's just throw our transformer on thi…

Another way to frame it is that these models still perform very poorly at the task they're designed to do. Imagine if real programmer needed to write a solution a hundred times before they were able to achieve (average) performance. You'd probably wonder if it was just blind luck that got them to the solution. You'd also fire them. What these models are very good at doing is plagiarizing content, so part of me wonder…

They specifically stated that they tested it on 10 challenges that were newer than their training data, so it couldn’t just be plagiarizing content.

Re: Competitive Programming with AlphaCode

#348

Post by zbobet2012 »

baobabKoodaa wrote: post #125
FemmeAndroid wrote: post #103

This is extremely impressive, but I do think it’s worth noting that these two things were provided: - a very well defined problem. (One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem, not something I experience on most days.) - existing test data. This is definitely a great accomplishment, but I think those two features of competitive program…

> One of the things I like about competitive programming and the like is just getting to implement a clearly articulated problem English versions of Codeforces problems may be well-defined but they are often very badly articulated and easy to misunderstand as a human reader. I still can't understand how they got AI to be able to generate plausible solutions from these problem statements.

They used the tests. The specification being very approximate is fine, because they had a prebuilt way to "check" if their result was good.

Re: Competitive Programming with AlphaCode

#349

Post by rabbits77 »

thephyber wrote: post #327
rabbits77 wrote: post #324

What I always find missing from these Deep Learning showcase examples are an honest comparison to existing work. It isn’t like computers haven’t been able to generate code before. Maybe the novelty here is working from the English language specification, but I am dubious just how useful that really is. Specifications are themselves hard to write well too. And what if the “specification” was some Lisp code testing a c…

I have fiddled with genetic programming. I don't think there is a good solution for a useful metric for comparing one code generator against another, so I don't think DeepMind should care. Most of the genetic programming results code generated by my algos doesn't compile. Very occasionally the random conditions exist to allow it to jump over a "local maxima" and come up with a useful candidate source code. Sometimes…

If you think tuning GP parameters are challenging wait until you try tuning hyper parameters for a DL model!

It is just a press release, to be fair to DeepMind, and I guess they can promote themselves however they wish.

My original comment was more from the context of seeing neural network models in practice perform barely any better, if at all, then classic ML models. Just as those comparisons were revealing similarly I was suspecting this use case may be the same to another classic technique.

GP is certainly not the shining star of AI right now but it is actively researched and perusing Google scholar on the subject will show you plenty of interesting, but less heralded, results.

There are probably several meaningful metrics for this problem that can be examined. If nothing else it is a simple matter of grading the solutions of each, like a university assignment. Also, typically classical techniques are less resource intensive then any neural network methods; the energy savings alone when considered at production scales would be significant.

Post reply on HN