Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AI based programming could work

bjenik.com

Post reply on HN
Print view View original

41–50 of 61 posts

Re: How AI based programming could work

#41

Post by ThePhysicist »

Very intersting ideas! I also think that in the future we will see a lot more declarative programming languages that model high-level concepts of a given program, while low-level "plumbing" code will be generated automatically. I'm not yet sure about the role that AI will play in this process though, and I think there are many things that we have to solve before we'll see something that can write generic programs.

Program synthesis is a very active field of research and many AI-based methods have been proposed in the last decades, IMHO what most systems lack is the applicability to real-world programming languages and use cases though.

Having worked on static program analysis, I know that even our current ability to understand and reason about existing programs is still very limited. A main reason for this is that most real-world systems are composed of many parts that are not easily specifiable under a single paradigm (e.g. templates, database code, configuration files).

To build a usable AI-based programming system, we will need:

* A description languge that is able to model ALL aspects of a given real-world system under a single paradigm

* A system to analyze and understand the artefacts produced by the above system

* A way to generate real-world code from the specification above, including a way to "fill in the blanks" that the user did not specify (as leaving out the details the whole point of such a system)

* A way to test the generated code against the specifications provided by the user and further "reasonable" assumptions, which will be needed as the specifications by the user will not be complete, see above

* A way to guide this process towards a reasonable program through user feedback within a reasonably small number of steps

While none of these things are impossible, implementing them is a significant challenge with a lot of unknowns for which we don't have good solutions yet.

I therefore think the first AI-based systems which we'll see in the coming years will be limited to specific problem domains (e.g. data analysis, logic programming) for which we can more easily build a system as the above.

Re: How AI based programming could work

#42

Post by brett40324 »

bjenik wrote: post #22
ww520 wrote: post #21

It's kind of vague. It would be better if he used a concrete example to illustrate how AI help programming.

Actually the examples in the text are quite concrete - the few lines would be everything you'd have to write to get such a site working (modulo a bit of http plumbing). Obviously we are nowhere near such a thing. The rest of the text acts as a bit of a pointer what could help to get there. From my other comment (in this case the example was an amazon like shopping site): > The thing I would like to get working is tel…

You're gonna have to make a clear distinction between web dev project configuration, and AI for making good choices that optimize meeting sales and marketing goals.

How is your 'project codebase markup' any different from filling in some radio buttons on a template generator or a CMS?

Re: How AI based programming could work

#43

Post by argonaut »

sapphireblue wrote: post #33
tiagoespinha wrote: post #25

Interesting idea, but to be honest I think it will never take off in a large scale. If you want to write a vague and generic piece of code that can figure out by itself what it is that its outputs ought to be with regards to its inputs, you are, with effect, creating artificial intelligence. This, in turn, requires teaching. What if suddenly you have new pages on your website? Your AI program thingy wouldn't be able…

3 years ago many systems that exist today would be regarded as "scifi" or "maybe 10 years into the future". One of the most striking examples is Visual Question Answering with dynamic memory networks [1]. The model is given raw pixels and encoded question in natural language, it should output a natural language answer. See how it works on data it hasn't seen: http://i.imgur.com/074G9pg.jpg Do not underestimate what g…

Then would know that visual question answering performs quite poorly. It makes for a nice demo, but it's trivial/quick/easy to find entire areas of questions where the models completely. The questions have to be worded very carefully. The models are quite finnicky and dumb. For example, dynamic memory nets aren't the state of the art - the state of the art for these datasets doesn't use anything nearly as complicated as memory, etc.

Re: How AI based programming could work

#44

Post by orasis »

arekkas wrote: post #34
orasis wrote: post #27

Earlier quoted context omitted.

I disagree with your conclusion. In my app, 7 Second Meditation, I am constantly authoring new content. It uses improve.ai to show that new content to some small number of users and learn how they react to that content. The learning happens from the users themselves. In my particular case, within about 300-500 views the system can tell if a new piece of content is total crap and will basically stop showing it. If the…

Your example isn't even real statistical/machine learning, it's basic A/B testing. This has nothing in common with orchestrating code.

I would contend that bayesian reinforcement learning is statistical machine learning.

The difference between A/B testing and reinforcement learning is that reinforcement learning does not use a control and does not wait for high confidence to continue to improve. This allows much more data-efficient learning than A/B testing because at every step it is making its best guess as to whether to continue to explore or exploit what its already learned.

As to it having nothing to do with orchestrating code:

Most code is logic reacting to data. Reinforcement learning replaces hand-coded logic with goal-driven logic. If you can use a declarative approach to specify your goals, much of the logic can be moved to reinforcement learning.

Re: How AI based programming could work

#45

Post by jcranmer »

The keyword you want to search for is "program synthesis." It already exists, and no neural nets need apply (neural nets tend to be useful only if you can't get anything else to work). As a bonus, it's not probabilistic like AI techniques tend to be but exact, based on SMT solvers and verified proof correctness.

An example of work showcased at this year's PLDI that's capable of doing this sort of stuff:

Fast synthesis of fast collections (https://homes.cs.washington.edu/~mernst/pubs/collection-synt...): specify a database-like data structure for a collection of objects and the list of queries performed on that data structure, and get out code that's as fast as hand-tuned data structures with fewer bugs.

Programmatic and Direct Manipulation, Together at Last (http://dl.acm.org/citation.cfm?id=2908103&dl=ACM&coll=DL&CFI...): Take an image generated by code (e.g., periodic stripes), and be able to manipulate that image by drag-and-drop, e.g., changing stripe size or period.

Stratified synthesis: automatically learning the x86-64 instruction set (https://stefanheule.com/publications/pldi16-strata/): 60% of the x86-64 instruction set can be formally specified starting from about 60 base or pseudo-instructions (basically, describing each instruction as an assembly program of simpler instructions).

Re: How AI based programming could work

#46

Post by jcoffland »

yazr wrote: post #35
jcoffland wrote: post #16

neural networks != AI

"How ML based programming will allow us to program using a higher-abstraction programming language" Better ?! There is also the famous quote about how any working (and understood) AI is no longer considered AI...

That quote will apply up until we encounter real AI.

Re: How AI based programming could work

#48

Post by YeGoblynQueenne »

>> I believe a lot of research today is limited by first looking at, and becoming an expert in, the status quo and then building small iterative improvements. It would be better to first find a goal and then looking for a way of getting there - at least that's the way we got to the moon.

Nope.

Re: How AI based programming could work

#50

Post by YeGoblynQueenne »

jcranmer wrote: post #45

The keyword you want to search for is "program synthesis." It already exists, and no neural nets need apply (neural nets tend to be useful only if you can't get anything else to work). As a bonus, it's not probabilistic like AI techniques tend to be but exact, based on SMT solvers and verified proof correctness. An example of work showcased at this year's PLDI that's capable of doing this sort of stuff: Fast synthesi…

Nice, thanks- particularly the x86-64 instruction set learning. I would kind of put that down as grammar induction rather than program synthesis though.
Post reply on HN