51–60 of 93 posts
Reposting the CS-2 teardown in case anyone missed it. The thermal and electrical engineering is absolutely nuts: https://vimeo.com/853557623 https://web.archive.org/web/20230812020202/https://www.youtu... (Vimeo/Archive because the original video was taken down from YouTube)
Earlier quoted context omitted.
20,000 amps 200,000 electrical contacts 850,000 cores and that's the "old" one. wow.
This is something I'm clueless about and can't really understand. They say this is 24kW hungry. How does CPU power consumption really work on electrical level, what warrants that much power, even for regular CPUs? Like, from the basics level.. is it resistance of the material with frequency of switching or what is really going on there? Where does the power go on such a relatively small surface? edit: thanks people,…
Each gate has logically two transistors of which exactly one is always conducting, either connecting the output to the supply rail making the output a one, or connecting the output to the ground rail making the output a zero. When the output of the gate changes, both transistors have to switch in order to connect the output to the other rail than before. While this happens both transistors are conducting at the same time allowing current to flow from the supply rail to the ground rail.
In addition to that the input capacitances of subsequent gates get charged from the supply rail when the output goes high and discharged into the ground rail when input goes low. So every signal change pumps some charge from the supply rail through the input capacitances to the ground rail.
Earlier quoted context omitted.
>> why just 44GB tho??? You can order one with 1.2 Petabytes of external memory. Is that enough? "External memory: 1.5TB, 12TB, or 1.2PB" https://www.cerebras.net/press-release/cerebras-announces-th... "214Pb/s Interconnect Bandwidth" https://www.cerebras.net/product-system/
I can't find the memory bandwidth to that external memory. Did they publish this?
Earlier quoted context omitted.
20,000 amps 200,000 electrical contacts 850,000 cores and that's the "old" one. wow.
This is something I'm clueless about and can't really understand. They say this is 24kW hungry. How does CPU power consumption really work on electrical level, what warrants that much power, even for regular CPUs? Like, from the basics level.. is it resistance of the material with frequency of switching or what is really going on there? Where does the power go on such a relatively small surface? edit: thanks people,…
Add up trillions of transistors, flicking on and off billions of times a second, and you get enormous power draws.
What is actually drawing power is the gate capacitance of the transistors. If the transistor were a physical switch, the gate capacitance is the "weight" that must be put on the switch to flip it. Of course this weight gets smaller as the switches shrink and as the tech improves, but it will always be non-zero.
None of this accounts for resistive losses either, which is just the cost of doing business for a CPU.
Earlier quoted context omitted.
24kW translates to like 32hp, so you could imagine this thing with a liquid cooling loop hooked up to something that looks like a car radiator.
24kW is on the lower end of a home heating gas boiler.
Better sell all nvidia stocks. Once these chips are common there is no need anymore for GPUs in training super large AI models.
Ex Cerebras engineer. In my opinion, this is not going to be the case. The WSE-2 was a b** to program and debug. Their compilation strategy is a dead end, and they invest very little into developer ease. My two cents.
Earlier quoted context omitted.
20,000 amps 200,000 electrical contacts 850,000 cores and that's the "old" one. wow.
This is something I'm clueless about and can't really understand. They say this is 24kW hungry. How does CPU power consumption really work on electrical level, what warrants that much power, even for regular CPUs? Like, from the basics level.. is it resistance of the material with frequency of switching or what is really going on there? Where does the power go on such a relatively small surface? edit: thanks people,…
Once you've done that, the transistor is on until the gate capacitor is discharged. This requires getting rid of the electrons you stuffed into it. The easiest is to just connect the gate to ground, essentially throwing the electrons away.
So for each time the transistor goes through an on-off cycle, you need to "spend" some electrons, which in turn need to be supplied from the power supply. Thus higher frequency means more current just from more on-off cycles per second.
There's also resistive losses and leakage currents and such.
Now in theory I suppose you could recycle some of these electrons (using a charge pump arrangement[3]), reducing the overall demand. But that would require relatively large capacitors, and on-chip capacitors take a lot of chip area which could have been used for many transistors instead.
[1]: https://en.wikipedia.org/wiki/CMOS
[2]: https://en.wikipedia.org/wiki/MOSFET#Modes_of_operation
According to the company, the new chip will enable training of AI models with up to 24 trillion parameters. Let me repeat that, in case you're as excited as I am: 24. Trillion. Parameters. For comparison, the largest AI models currently in use have around 0.5 trillion parameters, around 48x times smaller. Each parameter is a connection between artificial neurons . For example, inside an AI model, a linear layer that…
...
It's meaningless to say something can train a model that has 24 trillion parameters without specifying the dataset size and time it takes to train.
Better sell all nvidia stocks. Once these chips are common there is no need anymore for GPUs in training super large AI models.