When will computer hardware match the human brain? (1998)
1–10 of 186 posts
Re: When will computer hardware match the human brain? (1998)
#2Moravec's plots (from 1998!) are looking more and more like prophecies:
Re: When will computer hardware match the human brain? (1998)
#3Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup."
Do you agree?
I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the likes do not have a memory, so all prior interactions have to be re-submitted with each turn of a conversation).
Intuitively, I'd be more prone to ascribe intelligence based on convincing ways of construction that go along with intellignece-like performance, especially if the model also makes human-like errors.
Re: When will computer hardware match the human brain? (1998)
#4Quote from article
At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phenomenal growth must soon come to an end.
EDIT: This Wikipedia article puts an AMD CPU at ~2 million MIPS https://en.wikipedia.org/wiki/Instructions_per_second as the highest one.
Re: When will computer hardware match the human brain? (1998)
#5Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
M1/M2s are around 2-4 trillion if I’m not mistaken
Re: When will computer hardware match the human brain? (1998)
#6Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the…
Reducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some
intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w.
I highly doubt you'd be able to get even close to that kind of performance with current manufacturing processes. We'd need something entirely different from laser lithography for that to happen.
Re: When will computer hardware match the human brain? (1998)
#7Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
When the article was written, almost nobody cared about GPUs or similar architectures being really good at parallel computation.
So while he wrote about MIPS, and those have indeed basically stopped significantly improving, FLOPS have continued to improve. And for AI in particular, for inference at least, we can get away with 8 bit floats, which is why my phone does 1.58e13/second, only a factor of x60 from a million-billion.
Re: When will computer hardware match the human brain? (1998)
#8Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
Today we'd include the GPU. If we include supercomputers, we really have to compare total global capacity, which is on the order of 1 million times that of 1997 [0]. That basically fits his trend line.
[0]: https://ourworldindata.org/grapher/supercomputer-power-flops
Re: When will computer hardware match the human brain? (1998)
#9It's a tough question since we still don't understand the brain well enough to know what degree of fidelity of copying it is necessary to achieve the same/similar functionality.
What is the computational equivalent of a biological neuron? How much of the detailed chemistry is really relevant to it's operation (doing useful computation), or can we just use the ANN model of synapses as weights, and the neuron itself as a summation device plus a non-linearity?
Maybe we can functionally model the human brain at a much coarser, more efficient, level than individual neurons - at cortical mini/macro column level perhaps, or as an abstract architecture not directly related to nature's messy implementation?
Re: When will computer hardware match the human brain? (1998)
#10Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the…
I agree with Moravec. As he points out a bit later on:
> Only on the outside, where they can be appreciated as a whole, will the impression of intelligence emerge. A human brain, too, does not exhibit the intelligence under a neurobiologist's microscope that it does participating in a lively conversation.
We only have fuzzy definitions of "intelligence", not any essential, unambiguous things we can point to at a minute level, like a specific arrangement of certain atoms.
Put another way, we've used the term "intelligent" to refer to people (or not) because we found it useful to describe a complex bundle of traits in a simple way. But now that we're training LLMs to do things that used to be assumed to be exclusively the capacity of humans, the term is getting stretched and twisted and losing some of its usefulness.
Maybe it would be more useful to subdivide the term a bit by referring to "human intelligence" versus "LLM intelligence". And when some new developments in AI seem like they're different from "LLM intelligence", we can call them by whatever distinguishes them, like "Q* intelligence", for example.