Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’ new monster AI chip adds 1.4T transistors

spectrum.ieee.org

Post reply on HN
Print view View original

81–90 of 169 posts

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#81

Post by whimsicalism »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

I'm on board with you that there should be a "drop-in" cross support of these chips, but pytorch is at a way higher abstraction level than what should be commonly supported.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#82

Post by smithza »

wyxuan wrote: post #58
zitterbewegung wrote: post #39

Much better article from anandtech at https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...

Here's a video made by the author, Ian Cuttress, which goes into more detail as well. https://www.youtube.com/watch?v=FNd94_XaVlY

This is very cool. The explanation of how yield/defects was interesting: they can bypass cores with defects due to channeling and account for the statistical defects, allowing them to have a 100% yield.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#83

Post by socialdemocrat »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

Maybe people will get to their senses and switch to Julia instead of having to waste all these time on Python bindings.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#85

Post by NaturalPhallacy »

IshKebab wrote: post #27
etaioinshrdlu wrote: post #19

How much memory is on the chip, and what kind is it? Under what circumstances does the chip need to access external memory? What type of communication interfaces does this chip have? Also, if the chip is the size of a wafer, is it appropriate to call it a Chip?

It's 40 GB of SRAM. I doubt it supports external memory. > Also, if the chip is the size of a wafer, is it appropriate to call it a Chip? Good question. I think it is. I mean the word "chip" isn't really that well defined (is HBM one chip?), but given that they sell it as a single unit and you can't really cut it in half I think it's one chip.

Looking at the picture, the thing is a platter. Not a chip.

It's a really cool picture too.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#87

Post by meepmorp »

systemvoltage wrote: post #11
bob1029 wrote: post #9

Earlier quoted context omitted.

They probably aren't bothering to. The extreme economics for producing this type of chip are likely acceptable to the stakeholders. Also, there is no reason they cant have some redundancy throughout the design so you can fuse off bad parts. It all really depends on the nature of the anticipated vs actual defects, which is an extraordinarily deep rabbit hole to climb into.

I wonder how much this "chip" costs!

The previous generation was the $2-3 million range, and these are now in the neighborhood of "several million". Or, arm+leg.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#88

Post by habibur »

socialdemocrat wrote: post #83
sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

Maybe people will get to their senses and switch to Julia instead of having to waste all these time on Python bindings.

Python here basically works as a binder for C, which is what everything is written in.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#89

Post by zucker42 »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

> The philosophy here seems to be “if we build it, they’ll buy it.”

Supposedly Cerebras is already profitable, so it's hardly a situation where they are building something and hoping people buy it eventually.

> That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and with an inferencing pipeline to generate samples.

Again, based on the companies representations, Cebebras transparently supports Pytorch and Tensorflow, only requiring a few lines of changed code.

Source: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-... (Dr. Cutress's video on TechTechPotato is also good).

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#90

Post by frogblast »

claytonius wrote: post #45
typon wrote: post #26

Earlier quoted context omitted.

40GB of SRAM. Not quite big enough to fit the big models like GPT3

I don’t think it’s straightforward to do a head to head comparison. from: https://www.youtube.com/watch?v=yso2S2Svdlg @ 25:14 James Wang: "If a model doesn’t fit into a GPU’s HBM, is it smaller when it’s laid out in the Cerebras way relative to your 18 gigabytes?" Andrew Feldman: "It is — it’s smaller in that we hold different things in memory than they do. One can imagine a model that has more parameters than we can…

Are there any good resources out there describing in practice how existing training workloads are distributed among GPUs? (using tensorflow, pytorch, or whatever else?).

I'm curious how the problem effectively gets sliced.

Post reply on HN