121–130 of 158 posts
Earlier quoted context omitted.
By lowering the barrier of entry of programming further, I wonder if we'll see more bugs (like the price discount) as a result of this?
You still need a programmer to find the bugs. I think it's actually harder to spot and fix a bug, than to write a simple method that involves one.
So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…
If AI gets to the point where it has a reasonable understanding of the shape of the data & the basic spatial manipulations being applied (not far off IMO), I'd expect it to be waaaaaay better at discovering certain types of new algorithms than humans. It can handle thinking about algorithms that have millions of independently moving parts in a way a human can't.
Humans have the edge deriving algorithms that require a sequence of high-level steps on an abstraction. "Do this, then we get a thing, then we do some stuff to the thing, stretch it, squash it, massage it." AI sucks at that, it doesn't think in the same kind of flexible abstractions.
But imagine if you build an understanding of how the code will be compiled & how that will interact with the cache into the AI. That's very difficult for humans because you can't think about all those mechanics at once, we have to focus on one at a time. An AI that really gets it? I could see it writing a better sorting algorithm for a specific, complex datatype than a human could, or at the very least having the competetive edge because it can do it basically instantly.
That seems like it would be considerably more effective, because you're removing the noise/overhead of parsing the text and giving a much clearer model of what's being manipulated to the AI.
Earlier quoted context omitted.
I think you are underselling the potential of a model which deeply understand programming. Imagine combining such a model with something like AutoML-Zero: https://arxiv.org/abs/2003.03384 It may not be 'creative', but used as tab-completion, it's not being rewarded or incentivized or used in any way which would expose its abilities towards creating a new sort algorithm.
I agree on the tab-completion part. Something like Gmail's smart-compose could have potentially huge benefits here. But I'm not sure about the "deeply understand programming" part. Language modelling and "AI", in its current form, uncovers only statistical correlations and barely scratches the surface of what "understanding" is. This has restricted deployment of majority of academic research into the real-world and t…
This is recurrent and somewhat unfair. Current architectures have long known to be universal, capable of reproducing any computational structure (of finite depth for NNs, and Turing complete for RNNs); they have significant structural flexibility and in principle their learning can converge to "ideal processing structures" (which supposedly our brains also approach) given good enough training conditions (data, regimen, etc.). The network scales, timescales and dataset scale to achieve what comparable human function are debatable and unknown, but I believe it's very safe to judge them on function (this particular example is indeed quite impressive), because given their performance it's likely a powerful structure has emerged under the hood -- you can think of it emerging similarly to intelligence emerges from evolution (and of course human learning). Internal recurrent evaluations of logic and representations of language can all emerge.
I wouldn't describe this process as simply statistical inference, since it has complex computational priors and structure involved. It's really algorithmic learning.
Of course, you can bake in structure to accelerate this process, and we've been discovering very useful structures (such as CNNs, LSTMs, Transformer arch) which bias the models in the desired direction but still have internal flexibility.
So are we all going to be out of a job?
Programming languages are automation tools. Libraries. Frameworks. It should be pretty clear this is a lasting trend, and doesn't necessarily mean programmers will have more or less jobs (due to well known effects of automation such as enabling new applications and increasing demand from increased productivity). It does mean you probably need to keep learning to stay relevant, and use those tools to your advantage!
I'am a bit confused, is this built by OpenAI or Microsoft? Microsoft released the paper IntelliCode Compose: Code Generation Using Transformer [1] 4 days ago and there is no attribution to anyone from OpenAI in it. Are those two entirely separate and yet exactly similar initiatives? [1]: https://arxiv.org/abs/2005.08025v1
GPT-2 is built by OpenAI
So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't expect any ability for innovation. At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). That's not to say that it can't write new code,…
These are just baby steps, but holy shit is that impressive. It kind of feels like working with offshore devs, but it's in real time.