Cellular Automata as Convolutional Neural Networks
1–10 of 15 posts
Re: Cellular Automata as Convolutional Neural Networks
#2> We show that any CA may readily be represented using a convolutional neural network with a network-in-network architecture.
Re: Cellular Automata as Convolutional Neural Networks
#3See also: "learning" Conway's Game of Life configurations by gradient descent.
Re: Cellular Automata as Convolutional Neural Networks
#4Not sure I understand the importance of this...
Re: Cellular Automata as Convolutional Neural Networks
#5Not sure I understand the importance of this...
Indeed, CA can be represented by simple combinations of boolean functions, obviously by NN also, which is a combination of similar nonlinear functions.
Re: Cellular Automata as Convolutional Neural Networks
#6Not sure I understand the importance of this...
Indeed, CA can be represented by simple combinations of boolean functions, obviously by NN also, which is a combination of similar nonlinear functions.
A wide enough NN can represent any arbitrary binary function, but it's not obvious that one can learn it.
Re: Cellular Automata as Convolutional Neural Networks
#7Earlier quoted context omitted.
Indeed, CA can be represented by simple combinations of boolean functions, obviously by NN also, which is a combination of similar nonlinear functions.
A wide enough NN can represent any arbitrary binary function, but it's not obvious that one can learn it.
Yet the best NNs are deep, not wide.
Re: Cellular Automata as Convolutional Neural Networks
#8Earlier quoted context omitted.
A wide enough NN can represent any arbitrary binary function, but it's not obvious that one can learn it.
Yet the best NNs are deep, not wide.
What do you mean by 'the best'? Deeper architectures are popular because they quiet easy to train. They do work well in practice on many tasks (especially vision) but they have their limits.
Infinite wide networks are a newly active field and has recently shown some promising results, theoretically [1, 2] and empirically [3].
[1] https://arxiv.org/abs/2001.06931 [3] https://arxiv.org/abs/1806.07572 [2] https://ai.googleblog.com/2020/03/fast-and-easy-infinitely-w...
Re: Cellular Automata as Convolutional Neural Networks
#9See also: "learning" Conway's Game of Life configurations by gradient descent. https://hardmath123.github.io/conways-gradient.html
Excellent write up. I've coincidentally been experimenting with the same thing. Any idea whether this approach could be used to speed up a search for exact solutions?
Re: Cellular Automata as Convolutional Neural Networks
#10See also: "learning" Conway's Game of Life configurations by gradient descent. https://hardmath123.github.io/conways-gradient.html
I wonder if it is because using backpropagation all non-linear functions are chained together when the weights are learnt? Is it naive to think by that formulation the results will be quite similar since the final model equations are close?