Live data from Hacker News

How AI based programming could work

bjenik.com

11–20 of 61 posts

Re: How AI based programming could work

#11
The author proposes a lot of vague ideas in this article (for example "I believe one of the biggest problems is the use of Error Propagation and Gradient Descent") without references or any solid proofs why they are necessary to solve the proposed program (Automate programming using ML?).

In fact there is already a lot of solid work just on this subject:

* Learning algorithms from examples http://arxiv.org/abs/1511.07275 https://arxiv.org/abs/1410.5401

* Generating source code from natural language description http://arxiv.org/abs/1510.07211

* And, the most closest work to what author probably wants, a way to write a program in forth while leaving some functions as neural blackboxes to be learned from examples: http://arxiv.org/abs/1605.06640

* Also there is a whole research program by nothing less than Facebook AI Research that explicitly aims at creating a conversational AI agent that is able to translate user's natural language orders into programs (asking to the user additional questions if necessary): http://arxiv.org/abs/1511.08130 (there is also a summary here http://colinraffel.com/wiki/a_roadmap_towards_machine_intell... )

And deepmind is also working on conversational agents: https://youtu.be/vQXAsdMa_8A?t=1265

Given current success of such models, automating simple programming tasks maybe not as much research as engineering and scaling up problem.

There is a lot of exciting machine learning research out there nowadays. Almost all of this research is available for free from papers posted on arxiv. It is a really good idea to read more about state of the art before coming with new ideas.

Re: How AI based programming could work

#12

Suppose you have an AI that generated some code for you, but it doesn't do exactly what you want. Now try to debug it... what if programming wouldn't involve defining exact steps, but instead just roughly defining what we have and what we want and maybe giving a few hints, and having the computer generally do the right thing - wouldn't that be awesome? I experience enough frustration with things like debugging genera…

> and getting Google's search engine to find exactly what I want with its pseudo-AI machine-learning algorithms doing completely inexplicable things with my queries, that I think "generally do the right thing" is not a good idea.

Actually Google works kind of OK now after they overshot seriously some years ago. Today if you use doublequotes around the search terms it actually seems to search for that exact term.

Which I think is a good solution: reasonable AI with easy fallback when you need it.

I guess an AI VM would need an exact modifier: don't attempt any unspecified cleverness on this variable/method/class.

Specifying some kind of runtime assertions on steroids could perhaps also alleviate some problems.

Debugging could be simplified by having the VM outputting all its assumptions.

Re: How AI based programming could work

#13
Starts off with an interesting idea (programming is perfectly specified versus being underspecified but more efficient due to shared context) but then devolves significantly: neural networks are not the be all end all of smart computers.

Interesting part is that a smart computer could resolve ambiguities in a human description of program desired in a way similar to humans. Missing part to this idea is that unless it's perfect, it would need a way to explain choices and results and probably some dialog system to iteratively improve upon how it resolved ambiguities.

E.g, make me an app like Uber but for cats -> here is a version you can play with -> oh I don't really like the cat icon, can we change it -> sure, which of these would you like ; etc etc

Computer requires ever changing human context, and maybe could have individualized context. So it has to learn over time. The point is to maximize efficiency of getting things made: how little needs to be specified before computer gives you something you accept / how fast can you go from some vague idea in your head to working acceptable software.

You should at least be able to get as good as an arbitrarily large number of people who lived a very similar life to yours and were put in a time bubble where you could talk with them to have them write software for you, assuming they're organized very well and highly motivated and great programmers and all that.

Re: How AI based programming could work

#14
post #5
post #4

>what if programming wouldn't involve defining exact steps, but instead just roughly defining what we have and what we want and maybe giving a few hints, and having the computer generally do the right thing - wouldn't that be awesome? So, Prolog?

Why Prolog? I have never noticed that Prolog was any better than other computer languages at reading one's mind!

Generally doing "the right thing" based on a rough definition of "what we have and what we want" is not mind-reading, but it is essentially what Prolog does. The author seems to conflate AI with declarative programming here, and I think that was the point.

Re: How AI based programming could work

#15
I feel dumber after reading this article. So much, that my own human net of neurons can't figure out or explain why.

A partial explanation is that I dont like the assumption that once work is being performed at higher and higher abstracted layers, that machine hardware, OS kernels, compilers and linkers, package and dependency managers, and all other systems software supporting high level programming paradigms (including AI based methods) are going to fall in line at the application layer.

Re: How AI based programming could work

#17
post #5
post #4

>what if programming wouldn't involve defining exact steps, but instead just roughly defining what we have and what we want and maybe giving a few hints, and having the computer generally do the right thing - wouldn't that be awesome? So, Prolog?

Why Prolog? I have never noticed that Prolog was any better than other computer languages at reading one's mind!

Prolog, or at least logic programming languages, were thought to be the wave of the future of programming in the 80s. They, of course, turned out to be a massive bust.

Re: How AI based programming could work

#18

The author proposes a lot of vague ideas in this article (for example "I believe one of the biggest problems is the use of Error Propagation and Gradient Descent") without references or any solid proofs why they are necessary to solve the proposed program (Automate programming using ML?). In fact there is already a lot of solid work just on this subject: * Learning algorithms from examples http://arxiv.org/abs/1511.0…

The forth black box paper seems to be quite close actually. The big difference is that I would like to go for higher level tasks. The low level variant - having code missing a few lines/functions and filling those is indeed quite well researched. More interesting would be the opposite: Having blocks of code (called components in my text) and letting the net find a good way to use them. Normally the way a programmer would build something like let's say the amazon website is sketching up a few ideas how it could look - cutting it into small parts and then writing (or reusing) the code for the small parts.

A lot of the "using AI for programming" papers (AI in this case being anything from nets to logic stuff) focus on building these small parts - I'm more interested in the levels above that. The "generating code from natural language/dialogs" also kind of misses my idea as it still assumes a "programmer" to tell it what to do (in the amazon example "display the products on a grid" or "make the buy button bigger"). The programmer would in this case either guess the right thing ("everybody has their products in a grid") or run something like A/B testing against some metric ("items sold" or "$ revenue") and then go back to the conversational programming tool and tell it "change the button". Why not skip the programmer and give the metric directly to the net? The thing I would like to get working is telling the net "here is a ton of different buttons, grids, lists and a lot of other UI stuff, and there are all our products - I want to maximize revenue - do whatever you like to get there".

Regarding the vague ideas: I believe (also totally unsubstantiated - though research in this area would be quite interesting) that writing down ideas first and then looking into them (either by reading other papers or doing own research) is far better than the other way round, because it has a higher probability of leading to something new, because you don't follow assumptions and errors others potentially made which could lead to a dead end. Of course there is a certain trade off, because if you try to reinvent everything you will not get to the point of something new.

Re: How AI based programming could work

#19
Since the c language is already a human readable abstraction away from the actual hardware instructions, which then again are abstractions on what actually happens on the chip with the transistors and electrons, you could argue that programming in c is already pretty much 'talking to the computer human-style'.

As others said the use of frameworks and even higher level scripting languages with an ide is already a dumb-ai kind of human-friendly conversation with the machine.

A problem with driving this further down the road like the article suggests could be, that if i wanted to talk to the machine like i talk with other persons, the natural language used in this case migh not be very well suited to express what i actually want. I would need special definitions and terminology, and in some cases a general description would not suffice, so the need for precision in expression might stay.

The actual dialogue with the machine then might last as along as a programmer would have needed to program it. I could see how this might work for very simple programs (like for some uber-app which is not much more than a wrapper for a telephone call to the taxi central), but inherent complexity that other programs need cannot be simplified away by switching the language to a conversational chit chat level.

Re: How AI based programming could work

#20

The author proposes a lot of vague ideas in this article (for example "I believe one of the biggest problems is the use of Error Propagation and Gradient Descent") without references or any solid proofs why they are necessary to solve the proposed program (Automate programming using ML?). In fact there is already a lot of solid work just on this subject: * Learning algorithms from examples http://arxiv.org/abs/1511.0…

The first response of an actual AI confronted with the task of creating an Uber-for-cats would be: "What do you mean with that? Please be more precise."
Post reply on HN