71–80 of 158 posts
So are we all going to be out of a job?
Wow, this has the ability to be a total gamechanger. You have to be really observant about the bugs though, I would have totally missed the one with the price discount without executing it.
By lowering the barrier of entry of programming further, I wonder if we'll see more bugs (like the price discount) as a result of this?
I mean it is cool. But there is the thing, the natural description of a function is not always this unambiguous. When you are telling a function to 'compute XYZ', what you are actually doing is 'check whether X.a exists, if so execute branch 1), else branch 2)'. If the logic gets really complicated, then describing it accurately in human language isn't necessarily faster than doing it in code directly. Otherwise, we…
I agree that code generation of complex functions is hard. But I think the example given of unit testing - ie. natural language description of specific behavior of function -> code is extremely useful.
But that would require the condition on the type system, meaning the code-gen needs to understand the object's interface, which while not impossible in current techniques, but hard enough due computation complexity.
Again I don't dispute this tool being interesting. But claims it to be ground breaking or game changing is simply not right.
Majority of programmers time, is not typing down the code. It is to look at the comment/description, think about it, edit some code, then rethink then edit again.
This tool has potential to solve some typing time, but it still not going to things fundamentally.
Amazing! So the developer's role will shift to: 1) writing good enough descriptions of the code to be generated by the AI model 2) fixing any little issues in the generated code
Amazing! So the developer's role will shift to: 1) writing good enough descriptions of the code to be generated by the AI model 2) fixing any little issues in the generated code
that would be supervised ai training, until 3) programmers become obsolete.
So are we all going to be out of a job?
When AI can reliably convert business-speak into efficient bug-free code, everyone will be out of a job, because the business owners will ask it to write them another AI to replace every other task their business does.
How does this do compared to other models? Is this a totally cutting edge result? On the surface, it seems quite impressive, but sans an environment to try it out with, I cannot be entirely sure. Still, this does make me question whether I chose a safe career, haha. The thing is, I'd really need to see a live demo to see how good this is. Making mistakes is actually kind of a big issue; as most people know, debugging…
However, this is fairly impressive for a couple reasons. First, the system constructs programs from natural language descriptions, rather than examples of input-output pairs or a formal specification, which are the most common settings for program synthesis. Second, they're generating full blown python, not a smaller, domain specific language.
Finally, and this is pretty mind-blowing, is the seamless, idiomatic use of loops, branches, and function calls. I haven't seen previous program synthesis tools able to generate such complex code. They're typically limited to simple linear programs with less than about 100 lines. Complex control flow and function calls are still beyond their reach for the most part.
I'm not an active researcher in neural program synthesis, so my statements may not reflect the current state of the art.
I honestly thought that the most promising route forward for program synthesis would be a model that incorporated knowledge of the syntax and semantics of code. Most likely, a model that manipulated, or at least had some view of, the program's AST. This seems to be just throwing a giant Transformer model at github.
Fine tuning a vanilla language model on a giant corpus of code feels like a dead end for the field, long-term. It seems obvious to me that humans are doing something more than just statistical pattern recognition and generation when we write and reason about code.
Then again, it's hard to argue with results. I'm sure lots of pre-neural network voice recognition researchers were in love with the elegance of their hidden markov models.
Edit: Also, everyone should go try the FlashFill feature in Microsoft excel. As far as I know, it's the only example of program synthesis shipped in a consumer facing production system, and it works shockingly well.
As a product person wondering how much more productive this will make my engineers? In the surface looks impressive.
This tool is the programming equivalent of an AI writing scientific papers based on an abstract. It can follow all the formalities really well. It can write beautiful English sentences. It might write formulas or produce graphs. It will dot the i's and cross the t's.
But it's unclear whether what it says is actually correct and logically coherent when used in the main part of the paper (and not just for the introduction of the paper) or just pleasing-sounding nonsense.
It can certainly help make engineers look more productive, just like it could help someone write papers at record speed. Whether the results can/will have any deeper value is yet to be determined. Maybe it will just be used for the "boring" tasks - like the paper introduction.
My personal fear is that it will be very good at writing code that looks ok, even though there is a serious flaw. Essentially, programmers tend to become good at spotting irregularities in the code corresponding to common human errors. The mistakes of this AI might be much harder to spot because they don't stand out in the same way.