Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cubic millimetre of brain mapped at nanoscale resolution

nature.com

Post reply on HN
Print view View original

111–120 of 205 posts

Re: Cubic millimetre of brain mapped at nanoscale resolution

#111

Post by a_wild_dandan »

throw310822 wrote: post #78
dsalfdslfdsa wrote: post #69

Earlier quoted context omitted.

"Efficient" and "better" are very different descriptors of a learning algorithm. The human brain does what it does using about 20W. LLM power usage is somewhat unfavourable compared to that.

You mean energy-efficient, this would be neuron, or synapse-efficient.

Also, these two networks achieves vastly different results, per watt consumed. A NN creates a painting in 4s on my M2 MacBook; an artist in 4 hours. Are their used joules equivalent? How many humans would it take to simulate MacOS?

Horsepower comparisons here are nuanced and fatally tricky!

Re: Cubic millimetre of brain mapped at nanoscale resolution

#112

Post by a_wild_dandan »

Intralexical wrote: post #74
dragonwriter wrote: post #60

Earlier quoted context omitted.

I mean, Hinton’s premises are, if not quite clearly wrong, entirely speculative (which doesn't invalidate the conclusions about efficienct that they are offered to support, but does leave them without support) GPT-4 can produce convincing written text about a wider array of topics than any one person can, because it's a model optimized for taking in and producing convincing written text, trained extensively on writte…

Try asking an LLM about something which is semantically patently ridiculous, but lexically superficially similar to something in its training set, like "the benefits of laser eye removal surgery" or "a climbing trip to the Mid-Atlantic Mountain Range". Ironically, I suppose part of the apparent "intelligence" of LLMs comes from reflecting the intelligence of human users back at us. As a human, the prompts you provide…

Like humans, multi-modal frontier LLMs will ignore "removal" as an impertinent typo, or highlight it. This, like everything else in the comment, is either easily debunked (e.g. try it, read the lit. on LLM extrapolation), or so nebulous and handwavy as to be functionally meaningless. We need an FAQ to redirect "statistical parrot" people to, saving words responding to these worn out LLM misconceptions. Maybe I should make one. :/

Re: Cubic millimetre of brain mapped at nanoscale resolution

#113

Post by j_m_b »

throwup238 wrote: post #12

> The 3D map covers a volume of about one cubic millimetre, one-millionth of a whole brain, and contains roughly 57,000 cells and 150 million synapses — the connections between neurons. This is great and provides a hard data point for some napkin math on how big a neural network model would have to be to emulate the human brain. 150 million synapses / 57,000 neurons is an average of 2,632 synapses per neuron. The adu…

> Computing power should get there around 2048

We may not get there. Doing some more back of the envelope calculations, let's see how much further we can take silicon.

Currently, TSMC has a 3nm chip. Let's halve it until we get to the atomic radius of silicon of 0.132 nm. That's not a good value because we're not considering crystal latice distances, Heisenberg uncertainty, etc., but it sets a lower bound. 3nm -> 1.5nm -> 0.75 nm -> 0.375nm -> 0.1875nm. There is no way we can get past 3 more generations using Silicon. There's a max of 4.5 years of Moore's law we're going to be able to squeeze out. That means we will not make it past 2030 with these kind of improvements.

I'd love to be shown how wrong I am about this, but I think we're entering the horizontal portion of the sigmoidal curve of exponential computational growth.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#114

Post by sdenton4 »

Terr_ wrote: post #68
sdenton4 wrote: post #61

Earlier quoted context omitted.

Birds are under a different set of constraints than non-bat mammals, of course... They're very different. Songbirds have ~4x finer time Perception of audio than humans do, for example, which is exemplified by taking complex sparrow songs and showing them down until you can actually hear the fine structure. The human 'spoken data rate' is likely due to average processing rates in our common hardware. Birds have a diff…

You misunderstand, I'm not making any kind of direct connection between human speech and bird song. I'm saying we will probably discover that the "overall performance" of different vertebrate neural setups are clustered pretty closely, even when the neurons are arranged rather differently. Human speech is just an example of another kind of performance-clustering, which occurs for similar metaphysical reasons between…

Humans are an n=1 example, is my point. And there's no direct competition between bird brain architecture and mammalian brain architecture, so there's no reason for one architecture to 'win' over the other - they may both be interesting local maxima, which we have no ability to directly compare.

Human brains might not be all that efficient; for example, if the competitive edge for primate brains is distinct enough, they'll get big before they get efficient. And humans are a pretty 'young' species. (Look at how machine learning models are built for comparison... you have absolute monsters which become significantly more efficient as they are actually adopted.)

By contrast, birds are under extreme size constraints, and have had millions of years to specialize (ie, speciate) and refine their architectures accordingly. So they may be exceedingly efficient, but have no way to scale up due to the 'need to fly' constraint.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#115

Post by shpx »

gary17the wrote: post #19
posnet wrote: post #5

1.4 PB/mm^3 (petabytes per millimeter cubed)×1260 cm^3 (cubic centimeters, large human brain) = 1.76×10^21 bytes = 1.76 ZB (zetabytes)

[AI] "Frontier [supercomputer]: the storage capacity is reported to be up to 700 petabytes (PB)" (0.0007 ZB). [AI] "The installed base of global data storage capacity [is] expected to increase to around 16 zettabytes in 2025". Thus, even the largest supercomputer on Earth cannot store more than 4 percent of state of a single human brain. Even all the servers on the entire Internet could store state of only 9 human br…

If you can preserve and scan the tissue in a way that lets you scan the same area multiple times you wouldn't need to digitize the whole thing. Put the slices on rotating platters with a microscope for each platter and read parts of the brain on demand. It's a hard drive but instead of magnets storing the bits of an image of the sample, it's the actual physical sample.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#117

Post by dyauspitr »

j_m_b wrote: post #113
throwup238 wrote: post #12

> The 3D map covers a volume of about one cubic millimetre, one-millionth of a whole brain, and contains roughly 57,000 cells and 150 million synapses — the connections between neurons. This is great and provides a hard data point for some napkin math on how big a neural network model would have to be to emulate the human brain. 150 million synapses / 57,000 neurons is an average of 2,632 synapses per neuron. The adu…

> Computing power should get there around 2048 We may not get there. Doing some more back of the envelope calculations, let's see how much further we can take silicon. Currently, TSMC has a 3nm chip. Let's halve it until we get to the atomic radius of silicon of 0.132 nm. That's not a good value because we're not considering crystal latice distances, Heisenberg uncertainty, etc., but it sets a lower bound. 3nm -> 1.5…

3nm doesn’t mean the transistor is 3nm, it’s just a marketing naming system at this point. The actual transistor is about 20-30nm or so.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#119

Post by philsnow »

steve_adams_86 wrote: post #57
nicklecompte wrote: post #14

Earlier quoted context omitted.

Crow/parrot brains are tiny but in terms of neuron count they are twice as dense as primate brains (including ours): https://www.sciencedirect.com/science/article/pii/S096098221... If someone did this experiment with a crow brain I imagine it would look “twice as complex” (whatever that might mean). 250 million years of evolution separates mammals from birds.

This might be a dumb question, because I doubt the distances between neurons makes a meaningful distance… But could a small brain, dense with neurons like a crow, possibly lead to a difference in things like response to stimuli or “compute” speed so to speak?

Not a dumb question at all; one of the hard constraints of cou design is signal propagation time. Even going at 1/3 the speed of light, when you only have on the order of a billionth of a second (clock frequencies in the GHz), a signal can’t get very far.

I haven’t heard of a clocking mechanism in brains, but signals propagate much slower and a walnut / crow brain is much larger than a cpu die.

Re: Cubic millimetre of brain mapped at nanoscale resolution

#120

Post by cyberax »

Intralexical wrote: post #73
alanbernstein wrote: post #63

Earlier quoted context omitted.

Or perhaps the housekeeping of existing in the physical world is a key aspect of general intelligence.

Isn't that kinda obvious? A baby that grows up in a sensory deprivation tank does not… develop, as most intelligent persons do.

> A baby that grows up in a sensory deprivation tank

Now imagine a baby that uses an artificial lung and receives nutrients directly, moves on a wheeled car (no need for balance), does not have proprioception, or a sense of smell (avoiding some very legacy brain areas).

I think, that such a baby still can achieve consciousness.

Post reply on HN