Following the tradition, Google execs are going to dismiss this discovery as irrelevant to the ads business, and a couple years later when DLCA will have turned the world upside down, they'll try to take credit saying it's their employees who have made the discovery.
Differentiable Logic Cellular Automata
91–100 of 100 posts
Re: Differentiable Logic Cellular Automata
#92I wish we were all commenting about the ideas embedded in this paper. It intrigues me, but is out of my comfort zone. Love to read more content-related insights or criticisms rather than the long thread on the shamefully smooth, engaging, and occasionally rote style.
I was reminded immediately of Wolfram’s exploration of using cellular automata to get MNIST recognition results. The underlying mechanisms they both use are super different, but the ideas seem like strong siblings — I attach them in my mind as saying computational complexity is almost shockingly expressive, and finding ways to search around the space of computation is pretty powerful. That said, I put in like 4 minut…
Re: Differentiable Logic Cellular Automata
#93The Conway's game of life example isn't so impressive. The network isn't really reverse engineering rules, it's being trained on data that is equivalent to the rules. It's sort of like teaching + by giving it 400 data points triplets (a,b,c) with 1 <= a,b <= 20 and c = a + b.
Re: Differentiable Logic Cellular Automata
#94It’s interesting to see how differentiable logic/binary circuits can be made cheap at inference time. But what about the theoretical expressiveness of logic circuits vs baselines like MLPs? (And then of course compared to CNNs and other kernels.) Are logic circuits roughly equivalent in terms of memory and compute being used? For my use case, I don’t care about making inference cheaper (eg the benefit logical circuit…
Re: Differentiable Logic Cellular Automata
#95Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…
an edit -- a black and white checker board can be done in 5 gates. Conway was more like 350 in the paper, apologies!
Re: Differentiable Logic Cellular Automata
#96This is very interesting. I've been chasing novel universal Turing machine substrates. Collecting them like Pokémon for genetic programming experiments. I've played around with CAs before - rule 30/110/etc. - but this is a much more compelling take. I never thought to model the kernel like a digital logic circuit. The constraints of boolean logic, gates and circuits seem to create an interesting grain to build the fi…
Re: Differentiable Logic Cellular Automata
#97This also reminds me of using Hopfield networks to store images. Seems like Hopfield networks are a special case of this where the activation function of each cell is a simple sum, but I’m not sure. Another difference is that Hopfield networks are fully connected, so the neighborhood is the entire world, i.e., they are local in time but not local in space. Maybe someone can clarify this further?
Re: Differentiable Logic Cellular Automata
#98Late here, but a few comments: the main idea of the authors was to combine differential logic gates (an amazing invention I had not heard of) with cellular automata as they say in the paper, or more accurately I would say a grid topology of small neural networks (cells). The cells get and send information to their neighbors. The idea would be you create some sort of outcome for fitness (say an image you want the cell…
Re: Differentiable Logic Cellular Automata
#99This is very interesting. I've been chasing novel universal Turing machine substrates. Collecting them like Pokémon for genetic programming experiments. I've played around with CAs before - rule 30/110/etc. - but this is a much more compelling take. I never thought to model the kernel like a digital logic circuit. The constraints of boolean logic, gates and circuits seem to create an interesting grain to build the fi…
Re: Differentiable Logic Cellular Automata
#100I love playing around with cellular automata for doing art. It's amazing what kind of patterns can emerge (example: https://gods.art/math_videos/hex_func27l_21.html ). I may have to try to play with these DLCA.
Lovely! Thanks for sharing. Would these patterns keep generating indefinitely?