Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When will computer hardware match the human brain? (1998)

jetpress.org

Post reply on HN
Print view View original

81–90 of 186 posts

Re: When will computer hardware match the human brain? (1998)

#81

Post by smt88 »

hulitu wrote: post #66
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

Too many presumptions.

400 transistors per synapse is a foundational assumption and is total nonsense

Re: When will computer hardware match the human brain? (1998)

#83

Post by Terr_ »

smt88 wrote: post #81
hulitu wrote: post #66

Earlier quoted context omitted.

Too many presumptions.

400 transistors per synapse is a foundational assumption and is total nonsense

Also, at what rate? If it's not at least one second per second, that changes things quite a bit.

Re: When will computer hardware match the human brain? (1998)

#84

Post by rerdavies »

thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

> Blackwell GPU has 208 billion MOSFETs[3]

Things us hard sci fi fans will insist on:

- You have significantly undercounted transistors. As of 2024 you can put up to 8.4 terabytes of LPDDR5X memory into an NVidia Grace Blackwell rack. So that's 72 trillion transistors (and another 72 trillion capacitors) right there.

- A GPU executes significantly faster than a neuron.

- The hardware of one GPU can be used to simulate billions of neurons in realtime.

- Why limit yourself to one GB200 NVL72 rack, when you could have a warehouse full? (what happens when you create a mind that's a thousand times more powerful than a human mind?)

You really need to separate the comparison into state (how much memory), and computation rate. I think you'll find that an NVidia GB2000 NVL72 will outperform a brain by at least an order of magnitude. And the cost of feeding brains far exceeds the cost of feeding GB2000's. Plus, brains are notoriously unreliable.

The current generation of public-facing AIs are using ~24Gb of memory, mostly because using more would cost more than can conveniently given away or rented out for pennies. If I were an evil genius looking to take over the world today, I'd be building terabyte-scale AIs right now, and I'd not be telling ANYONE. And definitely not running it in Europe or the US where it might be facing imminent legislative attempts to limit what it can do. Antarctica, perhaps.

Re: When will computer hardware match the human brain? (1998)

#85

Post by glial »

ben_w wrote: post #23
jll29 wrote: post #3

Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the…

LLMs are radically unlike organic minds. Given their performance, I think it is important to pay attention to their weirdnesses — I can call them "intelligent" or "dumb" without contradiction depending on which specific point is under consideration. Transistors outpace biological synapses by the same degree to which a marathon runner outpaces continental drift . This speed difference is what allows computers to read…

> LLMs are radically unlike organic minds.

I used to think this too, but now I'm not so sure.

There's an influential school of thought arguing that one of the primary tasks of the brain is to predict sensory input, e.g. to take sequences of input and predict the next observation. This perspective explains many phenomena in perception, motor control, and more. In an abstract sense, it's not that different from what LLMs do -- take a sequence and predict the next item in a sequence.

The System 1 snd System 2 framework is appealing and quite helpful, but let's unpack it a little. A mode of response is said to be "System 1" if it's habitual, fast, and implemented as a stimulus-response mapping. Similar to LLMs, System 1 is a "lookup table" of actions.

System 2 is said to be slow, simulation-based, etc. But tasks performed via System 2 can transition to System 1 through practice ('automaticity' is the keyword here). Moreover, pre-automatic slow System 2 actions are compositions are simpler sets of actions. You deliberate about how to compose a photograph, or choose a school for your children, but many of the component actions (changing a camera setting, typing in a web URL) are habitual. It seems to me that what people call System 2 actions are often "stitched-together" System 1 behaviors. Solving calculus problem may be System 2, but adding 2+3 or writing the derivative of x^2 is System 1. I'm not sure the distinction between Systems 1 and 2 is as clear as people make it out to be -- and the effects of practice make the distinction even fuzzier.

What does System 2 have that LLMs lack? I'd argue: a working memory buffer. If you have working memory, you can then compose System 1 actions. In a way, a Turing machine is System 1 rule manipulation + working memory. Chain-of-thought is a hacky working memory buffer, and it improves results markedly. But I think we could do better with more intentional design.

[1] https://mitpress.mit.edu/9780262516013/bayesian-brain/ [2] https://pubmed.ncbi.nlm.nih.gov/35012898/

Re: When will computer hardware match the human brain? (1998)

#86

Post by idiotsecant »

hulitu wrote: post #66
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

Too many presumptions.

So list your better assumptions. The exercise is guesstimating the future. Gather your unique perspective and offer up some upper and lower bounds.

Re: When will computer hardware match the human brain? (1998)

#88

Post by idiotsecant »

hax0ron3 wrote: post #52
geysersam wrote: post #33

The estimate for how many operations per second the brain does is quite a wild guess. It starts out very reasonable, estimating the amount of information the eyes feed to the brain. But from there on it's really just a wild guess. We don't know how the brain processes visual input, we don't know what the fundamental unit of processing in the brain is, or if there is one.

It also blows my mind that that, even though DNA seems to just code for proteins and does not store a schematic for a brain in any way that we've been able to decipher so far, human eggs pretty reliably end up growing into people who 9 months after conception already have a bunch of stuff, including visual processing, working. Of course the DNA is not the only input, there is also the mother's body, the whole process…

It is awesome, in the full nearly dreadfully amazing sense of the word.
Post reply on HN