Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When will computer hardware match the human brain? (1998)

jetpress.org

Post reply on HN
Print view View original

121–130 of 186 posts

Re: When will computer hardware match the human brain? (1998)

#121

Post by tim333 »

smusamashah wrote: post #4

Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…

I'm not sure he predicts a billion mips. The graph tops out at a million mips per $1000. He figues brain equivalence is 100 million mips. If you assume mips are equivalent to flops then that's 100 teraflops. That would seem roughly to match:

>"NVIDIA GeForce RTX 4090 Is The First Gaming Graphics Card To Deliver 100 TFLOPs of Compute Performance" https://wccftech.com/nvidia-geforce-rtx-4090-is-the-first-ga...

Currently about $1,800 on Amazon.

Re: When will computer hardware match the human brain? (1998)

#122

Post by thangalin »

Retric wrote: post #111
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

Each of those cells has ~7,000 synapses each of which is both doing some computation and sending information. Further, this needs to be reconfigurable as synaptic connections aren’t static so you can’t simply make a chip with hardwired connections. You could ballpark that as that’s 400 * 7000 * 86 billion transistors to simulate a brain that can’t learn anything, though I don’t see the point. Reasonably equivalent re…

> Which requires splitting things across multiple chips and some kind of 3D mesh of really high bandwidth connections.

Agreed. In the novel, it's a neuromorphing perception lattice, to suggest a 3D mesh. If you'd like to have a read, I'd be grateful for your feedback.

Re: When will computer hardware match the human brain? (1998)

#123

Post by Retric »

tim333 wrote: post #114
thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

Transistors are much faster than synapses so a few can simulate a bunch of synapses. As a result you can probably get by with like a million times less than your estimate.

Your suggestion actually makes things much harder.

A single “cycle” (and you need ~100-1,000+ cycles per second) would involve ~7,000 synapses per neuron * 86 billion neurons random memory accesses each second. Each of those accesses need to first read the location of memory and then access that memory.

For perspective a video card which is heavily optimized for this only gets you low billions of random memory access per second. True random access across large amounts of memory is hard. Of course if you’re using a 1 million GPU supercomputer you now need to sync all this memory between each GPU every cycle, or have each GPU handle network traffic for all these memory requests…

PS: The article assumes more efficient AI designs for good reason, actually using a neuronal network the size of the brain in real time is prohibative.

Re: When will computer hardware match the human brain? (1998)

#124

Post by aeonik »

Retric wrote: post #123
tim333 wrote: post #114

Earlier quoted context omitted.

Transistors are much faster than synapses so a few can simulate a bunch of synapses. As a result you can probably get by with like a million times less than your estimate.

Your suggestion actually makes things much harder. A single “cycle” (and you need ~100-1,000+ cycles per second) would involve ~7,000 synapses per neuron * 86 billion neurons random memory accesses each second. Each of those accesses need to first read the location of memory and then access that memory. For perspective a video card which is heavily optimized for this only gets you low billions of random memory access…

Super computers are almost at this scale, if you go by FLOPS.

Distributed cloud systems certainly surpass this number as well. https://www.top500.org/lists/top500/2023/11/

Re: When will computer hardware match the human brain? (1998)

#125

Post by andoando »

rerdavies wrote: post #112
andoando wrote: post #30

Earlier quoted context omitted.

I am not a fan of this article. The vary foundation of computer science was an attempt to emulate a human mind processing data. Foundational changes are of course harder, but it does not mean we should drop it all together.

> The very foundation of computer science was an attempt to emulate a human mind processing data. The very foundation of computer science was an attempt to emulate a human mind mindlessly processing data. Fixed that for you. And I'm still not sure I agree. The foundation of computer science was at attempt to process data so that human minds didn't have to endure the drudgery of such mindless tasks.

Take a look at Turing words in his formulation of the Turing machine and I think it becomes quite clearly the man spent time thinking about what he is doing when he is doing computations.

The tape is a piece of paper, the head is the human, who is capable of reading data from the tape and writing to it. The symbols are discernible things on the paper, like numbers. The movement of the tape ("scanning") is the eyes going back and forth. At each symbol, the machine decides which rule to apply.

Its an inescapable fact that we are trying to get computers to 1. operate as close to how we think (as we are the ones who operate it) and 2. to produce results which resemble how we think.

Abstractions, inheritence, objects, etc are no doubt all heavily influenced by thinking about how we think. If we still programmed using 1s and 0s, we wouldnt be where we are.

It seems incredibly short sighted to me to believe that because a few decades of research hasnt panned out, that we should all together forget about it.

Re: When will computer hardware match the human brain? (1998)

#126

Post by akomtu »

thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

Simulating a 100 neuron nematode is still an unsolved problem.

Re: When will computer hardware match the human brain? (1998)

#127

Post by PaulDavisThe1st »

RaftPeople wrote: post #115
rerdavies wrote: post #84

Earlier quoted context omitted.

> Blackwell GPU has 208 billion MOSFETs[3] Things us hard sci fi fans will insist on: - You have significantly undercounted transistors. As of 2024 you can put up to 8.4 terabytes of LPDDR5X memory into an NVidia Grace Blackwell rack. So that's 72 trillion transistors (and another 72 trillion capacitors) right there. - A GPU executes significantly faster than a neuron. - The hardware of one GPU can be used to simulat…

> The hardware of one GPU can be used to simulate billions of neurons in realtime. You mean simulate billions artificial neurons, right? You can't simulate a single biological neuron yet because it's way too complex and they still don't even understand all of the details. If they did have all of the details, at minimum you would need to simulate the concentrations of ions internally and externally as well as the elec…

Depends a bit on how certain you are on your preferred answer to the question: is the brain analog or digital? If you think it is predominantly digital, quite a lot of the details of the biochemical mechanisms don't matter. If you think it is predominantly analog, they matter hugely.

Re: When will computer hardware match the human brain? (1998)

#128

Post by Retric »

aeonik wrote: post #124
Retric wrote: post #123

Earlier quoted context omitted.

Your suggestion actually makes things much harder. A single “cycle” (and you need ~100-1,000+ cycles per second) would involve ~7,000 synapses per neuron * 86 billion neurons random memory accesses each second. Each of those accesses need to first read the location of memory and then access that memory. For perspective a video card which is heavily optimized for this only gets you low billions of random memory access…

Super computers are almost at this scale, if you go by FLOPS. Distributed cloud systems certainly surpass this number as well. https://www.top500.org/lists/top500/2023/11/

Yea, FLOPS and RAM aren’t that problematic it’s really insane memory bandwidth or really novel hardware that’s at issue.

That or some novel AI architecture. I doubt we need to actually emulate a brain to reach near parity with human cognition.

Re: When will computer hardware match the human brain? (1998)

#129

Post by xattt »

thangalin wrote: post #37

2038. * 86 billion neurons in a brain [1] * 400 transistors to simulate a synapse [2] That's 34 trillion, 400 billion transistors to simulate a human brain. As of 2024, the GB200 Grace Blackwell GPU has 208 billion MOSFETs[3]. In 2023, AMD's MI300A CPU had 146 billion transistors[3]. In 2021, the Versal VP1802 FPGA had 92 billion transistors[3]. Intel projects 1 trillion by 2030; TSMC suggests 200 billion by 2030. We…

> We'll likely have real-time brain analogs by 2064.

When there’s occasional talk about triple-letter agencies having far-future tech now, I used to wonder what it might be. I guess this is one of those things.

Re: When will computer hardware match the human brain? (1998)

#130

Post by awesomeMilou »

ben_w wrote: post #29
awesomeMilou wrote: post #6

Earlier quoted context omitted.

Reducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w. I highly doubt you'd be able to get even close to that kind of performance with current manufacturing proce…

The problem isn't the manufacturing process, but rather the architecture. At a low level: We take an analog component, then drive it in a way that lets us treat it as digital, then combine loads of them together so we can synthesise a low-resolution approximation of an analog process. At a higher level: We don't really understand how our brains are architected yet, just that it can make better guesses from fewer exam…

> Also, 400 W of electricity is generally cheaper than 20 W of calories

Are you serious? I don't think you have to be an expert to see that the average human can perform more work per energy intake than the average GPU.

> The problem isn't the manufacturing process, but rather the architecture.

It's very much a problem, good luck trying to even emulate the 3D neural structure of the brain with lithography. And there are few other processes that can create structures at the required scale, with the required precision.

Post reply on HN