Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When will computer hardware match the human brain? (1998)

jetpress.org

Post reply on HN
Print view View original

151–160 of 186 posts

Re: When will computer hardware match the human brain? (1998)

#151

Post by janpmz »

RaftPeople wrote: post #68
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

> * 86 billion neurons in a brain [1]* > * 400 transistors to simulate a synapse [2]* > * That's 34 trillion, 400 billion transistors to simulate a human brain.* You forgot about: 1-Astrocytes - more common than neurons, computational, have their own style of internal and external signaling, have bi-directional communication with neurons at the synapse (and are thought to control the neurons) 2-Synapse: a single syna…

Reading about brain waves makes me think analog computing could be the way to go.

Re: When will computer hardware match the human brain? (1998)

#152

Post by atemerev »

At my workplace, we are now completing the multiscale simulation of a full mouse brain (neuron/synaptic activity, cellular metabolism, etc). It fits into our small supercomputer first deployed in 2019, and there's a large potential for simplification not yet realized.

I believe that a supercomputer from the top 5 can already simulate a human brain. But we don't have the data and the theory yet. We are developing it, though.

Re: When will computer hardware match the human brain? (1998)

#153

Post by atemerev »

rerdavies wrote: post #84
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

> Blackwell GPU has 208 billion MOSFETs[3] Things us hard sci fi fans will insist on: - You have significantly undercounted transistors. As of 2024 you can put up to 8.4 terabytes of LPDDR5X memory into an NVidia Grace Blackwell rack. So that's 72 trillion transistors (and another 72 trillion capacitors) right there. - A GPU executes significantly faster than a neuron. - The hardware of one GPU can be used to simulat…

1) There is not enough data in the entire world to train a trillion parameters large language model. And there is limited use for synthetic data, yet (we need an internal adversarial/evolutionary model like AlphaZero for that, which I think will be developed soon, but we are not there yet).

2) Models are not linearly scalable in terms of their performance. There are 8B parameter models (Zephyr-Mistral) routinely outperforming 180B parameter models (Falcon). Skill issue, like they say.

3) Even if you are an evil genius, you need money and computing power. Contrarily to what you might be reading in the media, there isn't much capital available to evil geniuses on a short notice (longer term arrangements might still be possible). And even if you have money, you can't just buy a truckload of H100s. Their sales are already highly regulated.

Re: When will computer hardware match the human brain? (1998)

#154

Post by sigmoid10 »

atemerev wrote: post #152

At my workplace, we are now completing the multiscale simulation of a full mouse brain (neuron/synaptic activity, cellular metabolism, etc). It fits into our small supercomputer first deployed in 2019, and there's a large potential for simplification not yet realized. I believe that a supercomputer from the top 5 can already simulate a human brain. But we don't have the data and the theory yet. We are developing it,…

Are you talking about real time or faster simulation of complete cellular metabolism incl. neural activity across an entire mouse brain? If not, then I'd say I can run a simulation like this on my home computer. It will just take forever to compute something meaningful. But if yes, that is insanely impressive, even though this won't scale to the human brain any time soon since it has 1000x more neurons and exponentially more synapses.

Re: When will computer hardware match the human brain? (1998)

#155

Post by 8bitsrule »

nullc wrote: post #139
hax0ron3 wrote: post #52

Earlier quoted context omitted.

It also blows my mind that that, even though DNA seems to just code for proteins and does not store a schematic for a brain in any way that we've been able to decipher so far, human eggs pretty reliably end up growing into people who 9 months after conception already have a bunch of stuff, including visual processing, working. Of course the DNA is not the only input, there is also the mother's body, the whole process…

newborns barely know how to see, they have to learn it. https://lozierinstitute.org/dive-deeper/the-newborn-senses-s...

Along with the native language they're surrounded with ... and have to progressively decode all the intricacies thereof.

Re: When will computer hardware match the human brain? (1998)

#156

Post by cornholio »

geysersam wrote: post #33

The estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the eyes feed to the brain. But from there on it's really just a wild guess. We don't know how the brain processes visual input, we don't know what the fundamental unit of processing in the brain is, or if there is one.

I'm going to go against the grain here and hypothesize that the computational requirements for emulating human reason are much lower than commonly claimed, and probably within the reach of a high end GPU released in the last/next 10 years.

If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans could traverse in centuries. Yet, their results are barely intelligent.

The human neural network, on the other hand, stores and is able to recall an estimated average of a few bits per waking second for a decade or two, barely a few GB of useful storage [1]. A great deal of effort and substance of the brain is dedicated to compressing the sensory input into an internal representation useful for rational though - a problem much simplified for a text-fed neural network. The ability of blind and deaf people (or even blind-deaf) for rational reasoning show these ancillary tasks are not intrinsic to human reason.

So things seem to indicate that what we lack is not hardware, but the right architectures and algorithms to unlock human level reason, the exceptionally tight self training loop of humans that can make sense of the world and even push humanity's understanding further using only a few GB of compressed training data.

[1] LANDAUER TK. How much do people remember ‐ some estimates of the quantity of learned information in long‐term‐memory.

Re: When will computer hardware match the human brain? (1998)

#158

Post by TheRoque »

cornholio wrote: post #156
geysersam wrote: post #33

The estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the eyes feed to the brain. But from there on it's really just a wild guess. We don't know how the brain processes visual input, we don't know what the fundamental unit of processing in the brain is, or if there is one.

I'm going to go against the grain here and hypothesize that the computational requirements for emulating human reason are much lower than commonly claimed, and probably within the reach of a high end GPU released in the last/next 10 years. If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans c…

> If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans couldn't traverse in centuries. Yet, their results are barely intelligent

Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any transformer has ingested. Moreover, words are already compressed data with a lot of lost information, nowhere near the raw experience about the world that our brains have. On top of that there's the evolutionary stuff.

Re: When will computer hardware match the human brain? (1998)

#159

Post by cornholio »

TheRoque wrote: post #158
cornholio wrote: post #156

Earlier quoted context omitted.

I'm going to go against the grain here and hypothesize that the computational requirements for emulating human reason are much lower than commonly claimed, and probably within the reach of a high end GPU released in the last/next 10 years. If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans c…

> If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans couldn't traverse in centuries. Yet, their results are barely intelligent Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any…

> Moreover, words are already compressed data with a lot of lost information

That's exactly the point, 99.9% of the information processed by the 6 year old has went into training his visual cortex, a subsystem not necessary for reasoning. This will eventually be used 8 hours a day for transforming visual representation of text on a computer screen into a compressed textual form which the inner rational loop can easily manipulate. The holy grail of AI, for anything but artistic creation, is just this inner loop.

Re: When will computer hardware match the human brain? (1998)

#160

Post by TheRoque »

cornholio wrote: post #159
TheRoque wrote: post #158

Earlier quoted context omitted.

> If you look at state of the art pre-trained transformers, they gobble up unfathomable quantities of dense textual data that far exceed what even the most learned and smart humans couldn't traverse in centuries. Yet, their results are barely intelligent Actually this is untrue. Yann Lecun himself said it. A kid at the age of 6 has had much much more data go through his brain (based on the visual data) than what any…

> Moreover, words are already compressed data with a lot of lost information That's exactly the point, 99.9% of the information processed by the 6 year old has went into training his visual cortex, a subsystem not necessary for reasoning. This will eventually be used 8 hours a day for transforming visual representation of text on a computer screen into a compressed textual form which the inner rational loop can easil…

This sounds like a huge oversimplification. A lot of knowledge doesn't come in the form of words, animals, early humans, kids, all don't have speech systems yet are capable of reasoning. You can't become Mozart just by reading the theory of his work, you can't become an expert at anything just by reading the theory in fact, even though it lifts you up. There's definitely things that you know and that aren't in the form of words.
Post reply on HN