Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Diffusion on syntax trees for program synthesis

tree-diffusion.github.io

Post reply on HN
Print view View original

21–30 of 95 posts

Re: Diffusion on syntax trees for program synthesis

#21

Post by elijahbenizzy »

29athrowaway wrote: post #10
pmayrgundter wrote: post #7

It's funny, this kind of subtree mutation was looked at pretty deeply by Koza and Adamı in the 90s under the rubric of Genetic Algorithms, but with a slightly different optimization function One ref in the paper to 2000 for GAs for fast generation of program trees, but that's missing the main show Hope they're reading this and dig into those guys work

You can also say backpropagation is the chain rule from centuries ago.

Backpropogration is just an application of the chain rule -- cool that we all learned it in high school!

Re: Diffusion on syntax trees for program synthesis

#22

Post by verdverm »

pmayrgundter wrote: post #7

It's funny, this kind of subtree mutation was looked at pretty deeply by Koza and Adamı in the 90s under the rubric of Genetic Algorithms, but with a slightly different optimization function One ref in the paper to 2000 for GAs for fast generation of program trees, but that's missing the main show Hope they're reading this and dig into those guys work

Some more recent alternatives to Koza's GP use some very different search mechanisms. FFX & PGE are both very fast.

https://seminars.math.binghamton.edu/ComboSem/worm-chiu.pge_...

https://arxiv.org/pdf/2209.09675

I authored PGE and have thought that RL, and more recently diffusion techniques, might help these algos. All of the algos need better ways to guide the search, or help it get unstuck from local optima, which happens surprisingly fast. Most work in GP / EVO is about avoiding premature convergence.

Re: Diffusion on syntax trees for program synthesis

#23

Post by samatman »

artninja1988 wrote: post #19

Surprised to see Stuart Russells name on this as I thought he was fully consumed by the doomsday cult. Although he's last author so he's probably only on it because he's the head of the lab

A lot of doomers work on AI. While frowning, and shaking their heads very gravely, so you know they don't approve.

Re: Diffusion on syntax trees for program synthesis

#24

Post by telotortium »

29athrowaway wrote: post #10
pmayrgundter wrote: post #7

It's funny, this kind of subtree mutation was looked at pretty deeply by Koza and Adamı in the 90s under the rubric of Genetic Algorithms, but with a slightly different optimization function One ref in the paper to 2000 for GAs for fast generation of program trees, but that's missing the main show Hope they're reading this and dig into those guys work

You can also say backpropagation is the chain rule from centuries ago.

It is a computationally clever application of the chain rule to minimize the amount of computation needed to compute gradients for all parameters in the network.

Re: Diffusion on syntax trees for program synthesis

#28

Post by revalo »

jmugan wrote: post #25

[flagged]

It's much easier to work with LISP. It's functional, trees are easy to reason about. We're using an even more simplified language in this work. It's kind of like the MNIST for progsynth research. Scaling this to real and useful programming languages and domains is still non-trivial, and is a major drawback of our work here.

Re: Diffusion on syntax trees for program synthesis

#29

Post by jmugan »

baq wrote: post #26
jmugan wrote: post #25

[flagged]

S-expressions are basically an AST with minimal parsing. It’s very convenient to work with.

Yeah, that's basically my objection. The convenience seems to hide something important. I know you can write any program with LISP, but using LISP for this kind of thing seems to be a symptom of limited applicability.

Re: Diffusion on syntax trees for program synthesis

#30

Post by ngruhn »

artninja1988 wrote: post #19

Surprised to see Stuart Russells name on this as I thought he was fully consumed by the doomsday cult. Although he's last author so he's probably only on it because he's the head of the lab

I haven’t heard anyone make sane case against the doomsday argument. Only attacks.
Post reply on HN