There are a lot of cool ideas here. Maybe a small observation but the computation is stateful. Each cell has a memory and perception of it's environment. Compare this to say your modern NN which are stateless. Has there been any work on statefull LLMs for instance?
Differentiable Logic Cellular Automata
31–40 of 100 posts
Re: Differentiable Logic Cellular Automata
#32This is exciting.
Michael Levin best posited for me the question of how animal cells can act cooperatively without a hierarchy. He has some biological experiments showing, for example, eye cells in a frog embryo will move to where the eye should go even if you pull it away. The question I don't think he could really answer was 'how do the cells know when to stop?'
Understanding non-hierarchical organization is key to understanding how society works, too. And to solve the various prisioner's delimmas at various scales in our self-organizing world.
It's also about understanding bare complexity and modeling it.
This is the first time I've seen the ability to model this stuff.
So many directions to go from here. Just wow.
Re: Differentiable Logic Cellular Automata
#33I've been thinking a lot about "intelligence" lately, and I feel like we're at a decisive point in figuring out (or at least greatly advance our understanding of) how it "works". It seems to me that intelligence is an emergent natural behavior, not much different than classical Newtonian mechanics or electricity. It all seems to boil down to simple rules in the end.
What if everything non-discrete about the brain is just "infrastructure"? Just supporting the fundamentally simple yet important core processes that do the actual work? What if it all boils down to logic gates and electrical signals, all the way down?
Interesting times ahead.
Re: Differentiable Logic Cellular Automata
#34There’s something compelling about these, especially w.r.t. their ability to generalize. But what is the vision here? What might these be able to do in the future? Or even philosophically speaking, what do these teach us about the world? We know a 1D cellular automata is Turing equivalent, so, at least from one perspective, NCA/these aren’t terribly suprising.
these are going to be the dominant lifeforms on earth exceeding bacteria, plants and humans in terms of energy consumption
cellular automata that interact with their environment, ones that interact with low level systems and high level institutions. to some approximation we, humans are just individual cells interacting in these networks. the future of intelligence aint llms, but systems of automata with metabolic aspects. automata that co-evolve, consume energy and produce value. ones that compete, ones that model each other.
we're not being replaced, we're just participants in a transformation where boundaries between technological and cellular systems blur and eventually dissolve. i'm very thankful to be here to witness it
Re: Differentiable Logic Cellular Automata
#35There’s something compelling about these, especially w.r.t. their ability to generalize. But what is the vision here? What might these be able to do in the future? Or even philosophically speaking, what do these teach us about the world? We know a 1D cellular automata is Turing equivalent, so, at least from one perspective, NCA/these aren’t terribly suprising.
these are going to be the dominant lifeforms on earth exceeding bacteria, plants and humans in terms of energy consumption cellular automata that interact with their environment, ones that interact with low level systems and high level institutions. to some approximation we, humans are just individual cells interacting in these networks. the future of intelligence aint llms, but systems of automata with metabolic asp…
I'll have what this guy is smoking. Those visualizations are pretty, though.
I can imagine this being useful for implementing classifiers and little baby GenAI-adjacent tech on an extremely tiny scale, on the order of several hundred or several thousand transistors.
Example: right now, a lot of the leading-edge biosensors have to pull data from their PPG/ECG/etc chips and run it through big fp32 matrices to get heart rate. That's hideously inefficient when you consider that your data is usually coming in as an int16 and resolution any better than 1bpm isn't necessary. But, fp32 is what the MCU can do in hardware so it's what you gotta do. Training one of these things to take incoming int16 data and spit out a heart rate could reduce the software complexity and cost of development for those products by several orders of magnitude, assuming someone like Maxim could shove it into their existing COTS biosensor chips.
Re: Differentiable Logic Cellular Automata
#36I wonder what Stephen Wolfram has to say about this.
[flagged]
Re: Differentiable Logic Cellular Automata
#37Is there any code available?
Re: Differentiable Logic Cellular Automata
#38Earlier quoted context omitted.
Which portion of the text gave you that impression?
> To answer this, we'll start by attacking Conway's Game of Life - perhaps the most iconic cellular automata, having captivated researchers for decades > At the heart of this project lies... > his powerful paradigm, pioneered by Mordvintsev et al., represents a fundamental shift in... (Not only is this clearly LLM-style, I doubt someone working in a group w/ Mordvintsev would write this) > Traditional cellular automa…
Reasearch papers are written like this and LLMs are trained on arxiv.
Re: Differentiable Logic Cellular Automata
#39Earlier quoted context omitted.
these are going to be the dominant lifeforms on earth exceeding bacteria, plants and humans in terms of energy consumption cellular automata that interact with their environment, ones that interact with low level systems and high level institutions. to some approximation we, humans are just individual cells interacting in these networks. the future of intelligence aint llms, but systems of automata with metabolic asp…
I'll have what this guy is smoking. Those visualizations are pretty, though. I can imagine this being useful for implementing classifiers and little baby GenAI-adjacent tech on an extremely tiny scale, on the order of several hundred or several thousand transistors. Example: right now, a lot of the leading-edge biosensors have to pull data from their PPG/ECG/etc chips and run it through big fp32 matrices to get heart…
yes absolutely: current systems are wildly inefficient. the future is one of extreme energy efficiency.
re smoking: sorry let me clarify my statement. these things will be the dominant life forms on earth in terms of metabolism, exceeding the energy consumption of biological systems, over 1k petawatt hours per year, dwarfing everything else
the lines betwen us may blur metaphorically, we'll be connected to them how we're connected to ecosystems of plants and bacteria. these systems will join and merge in the same way we've merged with smartphones -- but on a much deeper level
Re: Differentiable Logic Cellular Automata
#40Can anybody point out what's special about this?