Live data from Hacker News

Cubic millimetre of brain mapped at nanoscale resolution

nature.com

81–90 of 205 posts

Re: Cubic millimetre of brain mapped at nanoscale resolution

#81

Earlier quoted context omitted.

Or perhaps the housekeeping of existing in the physical world is a key aspect of general intelligence.

Isn't that kinda obvious? A baby that grows up in a sensory deprivation tank does not… develop, as most intelligent persons do.

A true sensory deprivation tank is not a fair comparison, I think, because AI is not deprived of all its 'senses' - it is still prompted, responds, etc.

Would a baby that grows up in a sensory deprivation tank, but is still able to communicate and learn from other humans, develop in a recognizable manner?

I would think so. Let's not try it ;)

Re: Cubic millimetre of brain mapped at nanoscale resolution

#82

Earlier quoted context omitted.

I mean, Hinton’s premises are, if not quite clearly wrong, entirely speculative (which doesn't invalidate the conclusions about efficienct that they are offered to support, but does leave them without support) GPT-4 can produce convincing written text about a wider array of topics than any one person can, because it's a model optimized for taking in and producing convincing written text, trained extensively on writte…

Try asking an LLM about something which is semantically patently ridiculous, but lexically superficially similar to something in its training set, like "the benefits of laser eye removal surgery" or "a climbing trip to the Mid-Atlantic Mountain Range". Ironically, I suppose part of the apparent "intelligence" of LLMs comes from reflecting the intelligence of human users back at us. As a human, the prompts you provide…

Do I need different prompts? These results seem sane to me. It interprets laser eye removal surgery as referring to LASIK, which I would do as well. When I clarified that I did mean removal, it said that the procedure didn't exist. It interprets Mid-Atlantic Mountain Range as referring to the Mid-Atlantic Ridge and notes that it is underwater and hard to access. Not that I'm arguing GPT-4 has a deeper understanding than you're suggesting, but these examples aren't making your point.

https://chat.openai.com/share/2234f40f-ccc3-4103-8f8f-8c3e68...

https://chat.openai.com/share/1642594c-6198-46b5-bbcb-984f1f...

Re: Cubic millimetre of brain mapped at nanoscale resolution

#83

Earlier quoted context omitted.

"Efficient" and "better" are very different descriptors of a learning algorithm. The human brain does what it does using about 20W. LLM power usage is somewhat unfavourable compared to that.

You mean energy-efficient, this would be neuron, or synapse-efficient.

I don't think we can say that, either. After all, the brain is able to perform both processing and storage with its neurons. The quotes about LLMs are talking only about connections between data items stored elsewhere.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#84
post #52

Earlier quoted context omitted.

Except you’d be missing the part that a neuron is not just a node with a number but a computational system itself.

I think you are missing the point. The calculation is intentionally underestimating the neurons, and even with that the brain ends up having more parameters than the current largest models by orders of magnitude. Yes the estimation is intentionally modelling the neurons simpler than they are likely to be. No, it is not “missing” anything.

The point is to make a ballpark estimate, or at least to estimate the order of magnitude.

From the sibling comment:

> Individual proteins are capable of basic computation which are then integrated into regulatory circuits, epigenetics, and cellular behavior.

If this is true, then there may be many orders of magnitude unaccounted for.

Imagine if our intelligent thought actually depends irreducibly on the complex interactions of proteins bumping into each other in solution. It would mean computers would never be able to play the same game.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#85
post #22

Annual reminder to re-read "There's plenty of room at the bottom" by Feynman. https://web.pa.msu.edu/people/yang/RFeynman_plentySpace.pdf Note the part where the biologists tell him to make an electron microscope that's 1000X more powerful. Then note what technology was used to scan these images.

I think it's actually "What you should do in order for us to make more rapid progress is to make the electron microscope 100 times better" and the state of art at the time was "it can only resolve about 10 angstroms" or I guess 1nm. So 100x better would be 0.1 angstrom / 0.01 nm.

We have made some progress it seems. Googling I see "up to 0.05 nm" for transmission electron microscopes and "less than 0.1 nanometers" for scanning. https://www.kentfaith.co.uk/blog/article_which-electron-micr...

For comparison the distance between hydrogen nuclei in H2 is 0.074 nm I think.

You can see the shape of molecules but it's still a bit fuzzy to see individual atoms https://cosmosmagazine.com/science/chemistry/molecular-model...

Re: Cubic millimetre of brain mapped at nanoscale resolution

#86

Earlier quoted context omitted.

You mean energy-efficient, this would be neuron, or synapse-efficient.

I don't think we can say that, either. After all, the brain is able to perform both processing and storage with its neurons. The quotes about LLMs are talking only about connections between data items stored elsewhere.

Stored where?

Re: Cubic millimetre of brain mapped at nanoscale resolution

#87
post #85
post #22

Annual reminder to re-read "There's plenty of room at the bottom" by Feynman. https://web.pa.msu.edu/people/yang/RFeynman_plentySpace.pdf Note the part where the biologists tell him to make an electron microscope that's 1000X more powerful. Then note what technology was used to scan these images.

I think it's actually "What you should do in order for us to make more rapid progress is to make the electron microscope 100 times better" and the state of art at the time was "it can only resolve about 10 angstroms" or I guess 1nm. So 100x better would be 0.1 angstrom / 0.01 nm. We have made some progress it seems. Googling I see "up to 0.05 nm" for transmission electron microscopes and "less than 0.1 nanometers" fo…

Resolution is only one aspect of EM that can be optimized.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#88

Another proof point that AGI is probably not possible. Growing actual bio brains is just way easier. Its never going to happen in silicon. Every machine will just have a cubic centimeter block of neuro meat embedded in it somewhere.

Hard disagree on this.

I strongly believe that there is a TON of potential for synthetic biology-- but not in computation.

People just forget how superior current silicon is for running algorithms; if you consider e.g. a 17 by 17 digit multiplication (double precision), then a current CPU can do that in the time it takes for light to reach your eye from the screen in front of you (!!!). During all the completely unavoidable latency (the time any visual stimulus takes to propagate and reach your consciousness), the CPU does millions more of those operations.

Any biocomputer would be limited to low-bandwidth, ultra high latency operations purely by design.

If you solely consider AGI as application, where abysmal latency and low input bandwidth might be acceptable, then it still appears to be extremely unlikely that we are going to reach that goal via synthetic biology; our current capabilities are just disappointing and not looking like they are gonna improve quickly.

Building artificial neural networks on silicon, on the other hand, capitalises on the almost exponential gains we made during the last decades, and already produces results that compare to say, a schoolchild, quite favorably; I'd argue that current LLM based approaches already eclipse the intellectual capabilities of ANY animal, for example. Artificial bio brains, on the other hand, are basically competing with worms right now...

Also consider that even though our brains might look daunting from a pure "upper bound on required complexity/number of connections" point of view, these limits are very unlikely to be applicable, because they confound implementation details, redundancy and irrelevant details. And we have precise bound on other parameters, that our technology already matches easily:

1) Artificial intelligence architecture can be bootstrapped from a CD-ROM worth of data (~700MiB for the whole human genome-- even that is mostly redundant)

2) Bandwidth for training is quite low, even when compressing the ~20year training time for an actual human into a more manageable timeframe

3) Operating power does not require more than ~20W.

4) No understanding was necessary to create human intelligence-- its purely a result of an iterative process (evolution).

Also consider human flight as an analogy: we did not achieve that by copying beating wings, powered by dozens of muscle groups and complex control algorithms-- those are just implementation details of existing biological systems. All we needed was the wing-concept itself and a bunch of trial-and-error.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#89

Earlier quoted context omitted.

I don't think we can say that, either. After all, the brain is able to perform both processing and storage with its neurons. The quotes about LLMs are talking only about connections between data items stored elsewhere.

Stored where?

You tell me. Not in the trillion links of a LLM, that's for sure.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#90
post #28

Earlier quoted context omitted.

I think you would also need the epigenetic side, which is very poorly understood: https://www.universityofcalifornia.edu/news/biologists-trans... We have more detail than this about the C. elegans nematode brain, yet we still no clue how nematode intelligence actually works.

How's OpenWorm coming along?

Badly: https://www.lesswrong.com/posts/mHqQxwKuzZS69CXX5/whole-brai... (the comments have some updates as of 2023)

Almost every other cell in the worm can be simulated with known biophysics. But we don't have a clue how any individual nematode neuron actually works. I don't have the link but there are a few teams in China working on visualizing brain activity in living C. elegans, but it's difficult to get good measurements without affecting the behavior of the worm (e.g. reacting to the dye).

Post reply on HN