Continuous relaxation of boolean algebra is an old idea with much literature. Circuit synthesis is a really well-researched field, with an annual conference and competition [1]. Google won the competition 2 years ago. I wonder if you have tried your learner against the IWLS competition data sets. That would calibrate the performance of your approach. If not, why not?
Differentiable Logic Cellular Automata
81–90 of 100 posts
Re: Differentiable Logic Cellular Automata
#82Earlier quoted context omitted.
Textbook models typically simulate normal development of an embryo, e.g. A-P and D-V (anterior-posterior and dorsal-ventral) patterning. The question Levin raises is how a perturbed embryo manages to develop normally, both "picasso tadpoles" where a scrambled face will re-organize into a normal face, and tadpoles with eyes transplanted to their tails, where an optic nerve forms across from the tail to the brain and a…
I'm not an expert on the actual biological mechanisms, but, it makes intuitive sense to me that both of those effects would occur in the situation you described from simple cells working on gradients: I was one of the authors on this paper during my undergrad[1] and the generalized idea of an eye being placed on a tail and having nerves routed successfully through the body via pheromone gradient is exactly the kind o…
very interesting, thanks for sharing.
Re: Differentiable Logic Cellular Automata
#83Could this be used to train an LLM? It seems the hidden states could be used to learn how to store history.
Re: Differentiable Logic Cellular Automata
#84Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…
Pruning excess gates will be interesting. I know this sort of thing generally works with reachability analysis, but I'm curious in practice how thorny this will be. Moreover I'm curious how "interpretable" the resulting circuits will be.
Either way this research is fantastic. What a result.
Re: Differentiable Logic Cellular Automata
#85Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…
an edit -- a black and white checker board can be done in 5 gates. Conway was more like 350 in the paper, apologies!
Re: Differentiable Logic Cellular Automata
#86Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…
Pruning excess gates will be interesting. I know this sort of thing generally works with reachability analysis, but I'm curious in practice how thorny this will be. Moreover I'm curious how "interpretable" the resulting circuits will be. Either way this research is fantastic. What a result.
For sure. I guess you could run static analysis on the gates to determine what "hits" and what doesn't -- I'm not a chip designer, but I know the tools are super sophisticated, and these are, ultimately, very small circuits.
I know that some early AI physics-enabled designs utilized "weird" analog features, but at small geometries especially, and in real life, everything is analog anyway. If these are gate-level, I guess the interpretability questions will be literally on assessing logic. There's so many paths to dig in here, it's super interesting.
Re: Differentiable Logic Cellular Automata
#87Earlier quoted context omitted.
thanks for catching this, the figure for the target was inverted when exporting for publication, corrected now.
Amazing paper, I re-read it in more detail today. It feels very rich, like almost a new field of study —- congratulations to the authors. I’m ninjaing in here to ask a q — you point out in the checkerboard initial discussion that the 5(!) circuit game of life implementation shows bottom left to top right bias — very intriguing. However, when you show larger versions of the circuit, and in all future demonstrations, t…
Author here. Thank you! You're seeing that correctly. The directional bias is the result of some initial symmetry breaking and likely random-seed dependent. The version that constructs the checkerboard from the top-right down was trained asynchronously, and the one from the bottom-left up was trained synchronously. The resulting circuits are different.
Re: Differentiable Logic Cellular Automata
#88This is a very interesting paper. Question though: it seems the cells gates since they're updated using a "global" gradient descent that it isn't truly parallel.
Is there any promise towards a strictly local weight adjustment method ?
Re: Differentiable Logic Cellular Automata
#89This writing feels so strongly LLM flavored. It's too bad, since I've really liked Alexander Mordvintsev's other work.
Are you sure you aren’t just falling into the “it’s all llm” trap? A lot of common writing styles are similar, and the most common ones are what LLMs imitate. I often am accused of llm writing. I don’t publish llm text because I think it is a social harm, so it’s pretty demoralising to have people call out my writing as “”llm slop”. OTOH, I have a few books published and people seem to find them handy, so there’s tha…
Don't take it too hard. I've seen someone accused of using an LLM to write something because they correctly used an oxford comma. It's definitely the trap.
Re: Differentiable Logic Cellular Automata
#90I love playing around with cellular automata for doing art. It's amazing what kind of patterns can emerge (example: https://gods.art/math_videos/hex_func27l_21.html ). I may have to try to play with these DLCA.
Reminds me of the old movie Andromeda Strain.