Live data from Hacker News

Differentiable Logic Cellular Automata

google-research.github.io

91–100 of 100 posts

Re: Differentiable Logic Cellular Automata

#91
This is a groundbreaking shit, and it's not about checkerboards or lizards. The Navier-Stokes differential equation governing fluid motion is an update rule that predicts the next state of any given point given its neighborhood and a few previous states. The key insight is that all the complexity of the fluid and air motion, the formation of clouds and the motion of flames, is governed by a simple law, by an equation that's uniformly and simultaneously applied to each point in space and all it needs is the immediate neighborhood of that point and its immediate history. Discovering this equation by looking at real-life samples is what's called science. It should be possible in principle to apply this DLCA model to a video recording of smoke, constrained to a thin 2D layer for simplicity, and let this model derive the Navier-Stokes equation. When you take this one step further and consider that the update rule itself may be changing according to another update rule, we'll get into some interesting territory. This might be why in our brain neurons are connected to a neighborhood of thousand neurons instead of just ten or so.

Following the tradition, Google execs are going to dismiss this discovery as irrelevant to the ads business, and a couple years later when DLCA will have turned the world upside down, they'll try to take credit saying it's their employees who have made the discovery.

Re: Differentiable Logic Cellular Automata

#92

I wish we were all commenting about the ideas embedded in this paper. It intrigues me, but is out of my comfort zone. Love to read more content-related insights or criticisms rather than the long thread on the shamefully smooth, engaging, and occasionally rote style.

I was reminded immediately of Wolfram’s exploration of using cellular automata to get MNIST recognition results. The underlying mechanisms they both use are super different, but the ideas seem like strong siblings — I attach them in my mind as saying computational complexity is almost shockingly expressive, and finding ways to search around the space of computation is pretty powerful. That said, I put in like 4 minut…

Thanks much!

Re: Differentiable Logic Cellular Automata

#93

The Conway's game of life example isn't so impressive. The network isn't really reverse engineering rules, it's being trained on data that is equivalent to the rules. It's sort of like teaching + by giving it 400 data points triplets (a,b,c) with 1 <= a,b <= 20 and c = a + b.

It wasn't meant to be much more than a sanity check, as I read it anyway.

Re: Differentiable Logic Cellular Automata

#94

It’s interesting to see how differentiable logic/binary circuits can be made cheap at inference time. But what about the theoretical expressiveness of logic circuits vs baselines like MLPs? (And then of course compared to CNNs and other kernels.) Are logic circuits roughly equivalent in terms of memory and compute being used? For my use case, I don’t care about making inference cheaper (eg the benefit logical circuit…

A MLP must be compilable to some arrangement of logic gates, so you could always try a tack like initializing everything as randomly-wired/connected MLPs, and perhaps doing some pretraining, before compiling to the logic gate version and training the logic gates directly. Or take the MLP random initialization, and imitate its distributions as your logic gate distribution for initialization.

Re: Differentiable Logic Cellular Automata

#95

Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…

an edit -- a black and white checker board can be done in 5 gates. Conway was more like 350 in the paper, apologies!

Yes, when I got to that part, I was unsure whether you actually need 350 logic gates to implement Conway's Game of Life. It feels like that cannot be the minimum number. But presumably other mechanisms already exist where we can automatically whittle down the number of logic gates necessary given a desired truth table.

Re: Differentiable Logic Cellular Automata

#96
post #28

This is very interesting. I've been chasing novel universal Turing machine substrates. Collecting them like Pokémon for genetic programming experiments. I've played around with CAs before - rule 30/110/etc. - but this is a much more compelling take. I never thought to model the kernel like a digital logic circuit. The constraints of boolean logic, gates and circuits seem to create an interesting grain to build the fi…

What a busy beaver you are.

Re: Differentiable Logic Cellular Automata

#97
This is very interesting! I think an exciting direction would be to arrive at minimal circuits that are to some extent comprehensible by humans. Now, this might not be possible for every system, but certainly the rules of Conway‘s GoL can be expressed in less than 350 logic gates per cell?

This also reminds me of using Hopfield networks to store images. Seems like Hopfield networks are a special case of this where the activation function of each cell is a simple sum, but I’m not sure. Another difference is that Hopfield networks are fully connected, so the neighborhood is the entire world, i.e., they are local in time but not local in space. Maybe someone can clarify this further?

Re: Differentiable Logic Cellular Automata

#98

Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…

Minecraft seems too big. But this looks like a thing that you could give few hours of tetris and it will spit out a working scheme for tetris.

Re: Differentiable Logic Cellular Automata

#99
post #28

This is very interesting. I've been chasing novel universal Turing machine substrates. Collecting them like Pokémon for genetic programming experiments. I've played around with CAs before - rule 30/110/etc. - but this is a much more compelling take. I never thought to model the kernel like a digital logic circuit. The constraints of boolean logic, gates and circuits seem to create an interesting grain to build the fi…

check out difflogic. differentiable neural net logic circuits that can be compiled to cuda or c code. their prototypical demo is an mnist classifier that can run at > 1M images/sec on cpu!

Re: Differentiable Logic Cellular Automata

#100
post #76
post #73

I love playing around with cellular automata for doing art. It's amazing what kind of patterns can emerge (example: https://gods.art/math_videos/hex_func27l_21.html ). I may have to try to play with these DLCA.

Lovely! Thanks for sharing. Would these patterns keep generating indefinitely?

Yes... well, usually. Sometimes the particles can all get annihilated, and then they stop repeating.
Post reply on HN