11–20 of 124 posts
Cerebras WSE-3 contains 4 trillion transistors and 8 exaflops per sec, 20 PB bandwidth. 62 times the cores of an H100.. 900,000. I wonder if the WSE-3 can compete on price / performance though. Interesting times!
And, 9 trillion flops per core in 4.4 million transistors per core. That sounds a bit too good to be true.
When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.
When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.
Computers are going to increase in size as they consume more and more power, requiring increasingly elaborate cooling systems. Like how the transistor made the big and hot vacuum tubes obsolete, maybe we’ll see some analog breakthrough do the same thing to transistors, at least for AI. I doubt there is a world where we use analog for general purpose computing, but it seems perfect for messy, probabilistic processes l…
NVIDIA just announced Blackwell which gets to 208bn transistors on a chip by stitching two dies together into a single GPU. https://www.nvidia.com/en-us/data-center/technologies/blackw...
They’re sticking two of them in a board with a Grace CPU in between, then linking 36 of those those boards together in racks with NVLink switches that offer “130TB/s of GPU bandwidth in one 72-GPU NVLink domain”.
In terms of marketing, NVidia calls one of those racks a GB200 NVL72 “super GPU”.
So on one level NVIDIA would say they already have a GPU with ‘trillions’ of transistors.
This article has two authors. One author is chairman of TSMC. The other author is Chief Scientist of TSMC. This is important to note because they clearly know some stuff and we should listen.
When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.
When the limits of digital reaches the boundary of physics, Analogue is going make a comeback. Human brain feels nearer to Analogue than Digital. I will be surprised to reach AGI without nearing the Order-of-Magnitude of brain processing. We need that ONE paper on analogue to end this quest of trillions and counting transistors.
I have some theories that this isn't necessary. 1.) Just because the brain is a general-purpose machine great at doing lots of things, doesn't mean it's great at each of those things. Like when two people are playing catch, and one of them sees the first fragments of a parabola and estimates where the ball is going to land- a computer can calculate that way more efficiently than a mind, despite the fact that both are quick enough to get the job done. 2.) While the brain is great at, say, putting names to faces... a good CV machine can do the job almost as well, and can annotate a video stream in real-time.
Combining 1.) the fact that some problems are much simpler to solve with classical algorithms instead of neural networking, and 2.) that many brain tasks can be farmed out to a coprocessor/service, my hypothesis is that the number of neurons/resources required to do the "secret sauce" part of agi could be greatly reduced.
Cerebras WSE-3 contains 4 trillion transistors and 8 exaflops per sec, 20 PB bandwidth. 62 times the cores of an H100.. 900,000. I wonder if the WSE-3 can compete on price / performance though. Interesting times!
Is anyone actually using those WSEs in anger yet? They're on their third generation now, but as far as I can tell the discussion of each generation consists of "Cerebras announces new giant chip" and then radio silence until they announce the next giant chip.
Earlier quoted context omitted.
Computers are going to increase in size as they consume more and more power, requiring increasingly elaborate cooling systems. Like how the transistor made the big and hot vacuum tubes obsolete, maybe we’ll see some analog breakthrough do the same thing to transistors, at least for AI. I doubt there is a world where we use analog for general purpose computing, but it seems perfect for messy, probabilistic processes l…
What's amazing is that the human brain does it all on the equivalent of like 20 watts of power. That's basically a couple of LED light bulbs.