11–20 of 186 posts
Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the…
Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor...
Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
With that in mind we are currently at 1 billion GigaFLOPS for the largest super computer in 2023. And the trendline since the 90's has kept up with the prediction made in the post. Although we can't exactly say 1 GigaFLOPS = 1000 MIPS.
https://ourworldindata.org/grapher/supercomputer-power-flops...
Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
An RTX 4090 has AI compute of up to 661 teraflops in FP16[1]. FLOPS and IPS are not interchangeable, but just for fun, that's 6.6 * 10^14 FLOPS, which is just short of 1 billion million (10^15).
1. https://www.tomshardware.com/features/nvidia-ada-lovelace-an...
Moravec's plots (from 1998!) are looking more and more like prophecies: https://www.jetpress.org/volume1/power_075.jpg https://www.jetpress.org/volume1/All_things_075.jpg
Quote from article At the present rate, computers suitable for humanlike robots will appear in the 2020s. Can the pace be sustained for another three decades? The graph shows no sign of abatement. If anything, it hints that further contractions in time scale are in store. But, one often encounters thoughtful articles by knowledgeable people in the semiconductor industry giving detailed reasons why the decades of phen…
M1/M2s are around 2-4 trillion if I’m not mistaken
George Hotz has a nice series of posts on this:
https://geohot.github.io/blog/jekyll/update/2020/08/20/a-sil...
https://geohot.github.io/blog/jekyll/update/2022/02/17/brain...
https://geohot.github.io/blog/jekyll/update/2023/04/26/a-per...
I sorted them by date.
Moravac (in the linked paper): "In both cases, the evidence for an intelligent mind lies in the machine's performance, not its makeup." Do you agree? I'm much less keen to ascribe "intelligence" to large, pretrained language models given that I know how primitive their training regime is compared to a scenario where I might have been "blended" by their ability to "chat" (double quote here since I know ChatGPT and the…
Reducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w. I highly doubt you'd be able to get even close to that kind of performance with current manufacturing proce…
Like, you know . . . nerve cells.
Earlier quoted context omitted.
Reducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w. I highly doubt you'd be able to get even close to that kind of performance with current manufacturing proce…
> We'd need something entirely different from laser lithography for that to happen. Like, you know . . . nerve cells.
Aren't we there yet? George Hotz has a nice series of posts on this: https://geohot.github.io/blog/jekyll/update/2020/08/20/a-sil... https://geohot.github.io/blog/jekyll/update/2022/02/17/brain... https://geohot.github.io/blog/jekyll/update/2023/04/26/a-per... I sorted them by date.