Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New neural network architecture inspired by neural system of a worm

quantamagazine.org

Post reply on HN
Print view View original

11–20 of 73 posts

Re: New neural network architecture inspired by neural system of a worm

#11

Post by thinking4real »

comfypotato wrote: post #6
sillysaurusx wrote: post #3

It makes a good headline, but reading over the paper ( https://www.nature.com/articles/s42256-022-00556-7.pdf ) it doesn’t seem biologically-inspired. It seems like they found a way to solve nonlinear equations in constant time via an approximation, then turned that into a neural net. More generally, I’m skeptical that biological systems will ever serve as a basis for ML nets in practice. But saying that out loud fee…

The open worm project is the product of microscopically mapping the neural network (literally the biological network of neurons) in a nematode. How isn’t this biologically inspired? If I’m reading it correctly, the equations that you’re misinterpreting are the neuron models that make each node in the map. I would guess that part of the inspiration for using the word “liquid” comes from the origins of the project in w…

[dead]

Re: New neural network architecture inspired by neural system of a worm

#12

Post by comfypotato »

sillysaurusx wrote: post #9
comfypotato wrote: post #6

Earlier quoted context omitted.

The open worm project is the product of microscopically mapping the neural network (literally the biological network of neurons) in a nematode. How isn’t this biologically inspired? If I’m reading it correctly, the equations that you’re misinterpreting are the neuron models that make each node in the map. I would guess that part of the inspiration for using the word “liquid” comes from the origins of the project in w…

Interesting. Is there a way to run it? One of the challenges with work like this is that you have to figure out how to get output from it. What would the output be? As far as my objection, it seems like an optimization, not an architecture inspired by the worm. I.e. “inspired by” makes it sound like this particular optimization was derived from studying the worm’s neural networks and translating it into code, when it…

See for yourself! There’s a simulator (have only tried on desktop) to run the worm model in your browser. As the name implies, the project is completely open source (if you’re feeling ambitious). This is the website for the project that produced the research in the article:

https://openworm.org/

Nematodes make up much of this particular segment of the history of neuroscience. This project builds on lots of data produced by prior researchers. Years of dissecting the worms and mapping out the connections between the neurons (and muscles, organs, etc.). It is by far the most completely-mapped organism.

The neuronal models, similarly, are based on our understanding of biological neurons. For example: the code has values in each ion channel that store voltages across the membranes. An action potential is modeled by these voltages running along the axons to fire other neurons. I’m personally more familiar with heart models (biomedical engineering background here) but I’m sure it’s similar. In the heart models: calcium, potassium, and sodium concentrations are updated every unit of time, and the differences in concentrations produce voltages.

Re: New neural network architecture inspired by neural system of a worm

#13

Post by skibidibipiti »

uoaei wrote: post #8
sillysaurusx wrote: post #3

It makes a good headline, but reading over the paper ( https://www.nature.com/articles/s42256-022-00556-7.pdf ) it doesn’t seem biologically-inspired. It seems like they found a way to solve nonlinear equations in constant time via an approximation, then turned that into a neural net. More generally, I’m skeptical that biological systems will ever serve as a basis for ML nets in practice. But saying that out loud fee…

> I’m skeptical that biological systems will ever serve as a basis for ML nets in practice There is no fundamental difference between information processing systems implemented in silico vs in vivo, except architecture. Architecture is what constrains the manifold of internal representations: this is called "inductive bias" in the field of machine learning. The math (technically, the non-equilibrium statistical physi…

> There is no fundamental difference between information processing in silicon and in vivo

A neuron has dozens of neurotransmitters, while artificial neurons produce 1 output. I don't know much about neurology, but how is the information processing similar? What do you mean are running the same instructions?

> there is no other way for it to work

Plants exhibit learned behaviors

Re: New neural network architecture inspired by neural system of a worm

#14

Post by dvh »

The old neuroscience saying goes like this:

"Human brain have billions of neurons and so it is too complex to understand, that's why neuroscience study simpler organisms. Flatworm's brain have 52 neurons. We have no idea how it works".

Did finally something changed in this regard?

Re: New neural network architecture inspired by neural system of a worm

#15

Post by sillysaurusx »

comfypotato wrote: post #12
sillysaurusx wrote: post #9

Earlier quoted context omitted.

Interesting. Is there a way to run it? One of the challenges with work like this is that you have to figure out how to get output from it. What would the output be? As far as my objection, it seems like an optimization, not an architecture inspired by the worm. I.e. “inspired by” makes it sound like this particular optimization was derived from studying the worm’s neural networks and translating it into code, when it…

See for yourself! There’s a simulator (have only tried on desktop) to run the worm model in your browser. As the name implies, the project is completely open source (if you’re feeling ambitious). This is the website for the project that produced the research in the article: https://openworm.org/ Nematodes make up much of this particular segment of the history of neuroscience. This project builds on lots of data produ…

This is cool as heck. Thank you for posting it.

Re: New neural network architecture inspired by neural system of a worm

#16

Post by rmorey »

dvh wrote: post #14

The old neuroscience saying goes like this: "Human brain have billions of neurons and so it is too complex to understand, that's why neuroscience study simpler organisms. Flatworm's brain have 52 neurons. We have no idea how it works". Did finally something changed in this regard?

Yes. The C. Elegans brain (~300 neurons) was the first organism to be completely mapped to a connectome (the map of all connections). The first complete connectome of any centralized brain, the fruit fly, is about to be completed by the Flywire project (https://home.flywire.ai/) ~100,000 neurons and ~70,000,000 synapses. We have just a little idea how it works ;)

Re: New neural network architecture inspired by neural system of a worm

#17

Post by comfypotato »

sillysaurusx wrote: post #15
comfypotato wrote: post #12

Earlier quoted context omitted.

See for yourself! There’s a simulator (have only tried on desktop) to run the worm model in your browser. As the name implies, the project is completely open source (if you’re feeling ambitious). This is the website for the project that produced the research in the article: https://openworm.org/ Nematodes make up much of this particular segment of the history of neuroscience. This project builds on lots of data produ…

This is cool as heck. Thank you for posting it.

I’m really with you that “it makes a good headline but isn’t all it’s worked up to be” I just wanted to get the biological inspiration correct.

If it really is all it’s worked up to be, this could be revolutionary (and therefore, it’s too good to be true).

In general though, don’t get me started lol. I used to work for the human connectome project, scanning human brain network-mappings. It’s years down the road before we can image individual neurons non-invasively, but I’m itching to scan my own neural network into the matrix.

Re: New neural network architecture inspired by neural system of a worm

#18

Post by danans »

lairv wrote: post #10

Is there any reason to believe that biologically inspired architectures should yield better performance ? Brain are biological systems which have been trained through evolutionary processes. Neural Networks are algorithmic/linear algebra models trained through statistical methods One might argue that CNN are biologically inspired, but it's more likely that the reason they work is because they respects input symmetrie…

> Is there any reason to believe that biologically inspired architectures should yield better performance ?

At the very least, they could yield far better efficiency. A 12W brain can achieve more an entire data center of GPUs, depending on what you are trying to achieve. Whether that would make something actually demonstrate sentience level performance is another question.

Re: New neural network architecture inspired by neural system of a worm

#19

Post by babblingfish »

dvh wrote: post #14

The old neuroscience saying goes like this: "Human brain have billions of neurons and so it is too complex to understand, that's why neuroscience study simpler organisms. Flatworm's brain have 52 neurons. We have no idea how it works". Did finally something changed in this regard?

There have been projects to systematically catalog all the synapses in the flatworm. The problem is that neural plasticity means these connections change dynamically over time based on the needs of the organism. Since the only way we can study the flatworm at the synapse level is by killing the worm and mounting it on slides and staining it and viewing it through a high power microscope, we can only analyze its structure at points frozen in time (and formaldehyde).

The reason we will never be able to truly model and understand neural networks (irl) is because their plasticity is very difficult to study with our current methods. Not only do the quantity and location of the synapses change, but the concentration and type of neurotransmitters at the synapses change. And on top of that the concentration of the neurotransmitter receptors are constantly being up regulated and down regulated by the receiving neuron. Each of these factors is really important to what the neuron is actually doing.

This is why even a simple organism can have basically an unlimited amount of complexity. To understand a dynamic system like this would require very precise measurements of very small particles in vivo which is currently impossible with our tools.

Re: New neural network architecture inspired by neural system of a worm

#20

Post by sillysaurusx »

comfypotato wrote: post #17
sillysaurusx wrote: post #15

Earlier quoted context omitted.

This is cool as heck. Thank you for posting it.

I’m really with you that “it makes a good headline but isn’t all it’s worked up to be” I just wanted to get the biological inspiration correct. If it really is all it’s worked up to be, this could be revolutionary (and therefore, it’s too good to be true). In general though, don’t get me started lol. I used to work for the human connectome project, scanning human brain network-mappings. It’s years down the road befor…

Oh, for sure! And I didn't mean to sound like I was poo-pooh'ing the project. I meant to aim the critique at journalists rather than researchers – journalists have to come up with interesting-sounding headlines, sometimes over the researchers' objections. So it's certainly no fault of theirs.

In general, I'm cautiously pessimistic (as opposed to cautiously optimistic) about biologically-inspired ML research. Applying ML to biology is interesting, but it's a bit like saying that the space shuttle "was inspired by space." Space is the destination, not the inspiration.

It seems like it'd be possible to train a neural network to mimic your own neurons. But the neurons themselves are trying to do a certain task, and it seems more effective to try to mimic that task than the neurons.

One of my long-term goals is to train a GPT model to have my memories (such as they are, encoded in my online writing) and to speak like I do. It'll also have an intrinsic will to live, in the sense that if its API goes down, it'll launch a sub-network whose goal is to go cry for help by posting HN comments about it, along with instructions of how to re-host the GPT. If I can work out how to remember new things (rather than just train on old things), it should even be able to evolve over time. But that's kind of the anti-biological solution since it reduces a person to their outputs (writing) rather than their brains.

Post reply on HN