Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’ new monster AI chip adds 1.4T transistors

spectrum.ieee.org

Post reply on HN
Print view View original

141–150 of 169 posts

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#141

Post by dnautics »

socialdemocrat wrote: post #83
sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

Maybe people will get to their senses and switch to Julia instead of having to waste all these time on Python bindings.

Honestly I think julia missed the boat. There was an opportunity for julia to treat the gpu as an abstract distributed node (which is a first class concept in julia) and pay respect to the reality of data gravity. But they chose to instead basically treat the gpu as a synchronous entity.

I'm now bullish on elixir-nx because I think there's an outside shot they will get it right.

Python is a lost cause.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#142

Post by fromaj »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

But they do have PyTorch support??? They ship a 'cerebras graph compiler' extension for pytorch that allows you to replace the default optimizer with theirs.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#143

Post by virtuallynathan »

xhrpost wrote: post #35

How can the chip itself consume that kind of power? Or is the 15kw value for the entire unit? That's like 10 residential space heaters all turned to max. I'm surprised that much heat could be dissipated over such a small surface area. Does it use refrigerant for cooling? If my math is correct, if you had a 6500BTU window air conditioner, you'd need 8 of them to move the heat from this chip.

It’s water cooled for sure, perhaps with chilled water, but all integrated into the 15U chassis. There are cutaway images of the v1 system online.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#144

Post by byronyi »

sillysaurusx wrote: post #139
solidasparagus wrote: post #137

Earlier quoted context omitted.

I don't really agree, at least from a longer term perspective. It's early days yet, but XLA seems to be a promising intermediate representation for allowing the DL frameworks to run on a wider array of hardware without user-facing software changes. It has traction with Google, NVIDIA, IIRC Intel and maybe more (others are definitely using the same approach of compute graph splitting and subgraph scheduling, but I'm n…

Mr/mrs anonymous HN person, please put some info in your profile. You clearly have some deep knowledge of TPUs that I didn’t expect to pop up offhandedly on HN. You’re correct on all counts: dynamic tensor shapes are more or less impossible with XLA, making it more or less impossible to train a model with arbitrary image size inputs, even though the math would allow for that; the pytorch XLA work on TPUs is indeed ki…

+1 please put some info in your profile.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#145

Post by sillysaurusx »

byronyi wrote: post #144
sillysaurusx wrote: post #139

Earlier quoted context omitted.

Mr/mrs anonymous HN person, please put some info in your profile. You clearly have some deep knowledge of TPUs that I didn’t expect to pop up offhandedly on HN. You’re correct on all counts: dynamic tensor shapes are more or less impossible with XLA, making it more or less impossible to train a model with arbitrary image size inputs, even though the math would allow for that; the pytorch XLA work on TPUs is indeed ki…

+1 please put some info in your profile.

I'm dying to know what they work on, either officially or in their spare time. https://news.ycombinator.com/item?id=26586151

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#146

Post by IshKebab »

NaturalPhallacy wrote: post #85
IshKebab wrote: post #27

Earlier quoted context omitted.

It's 40 GB of SRAM. I doubt it supports external memory. > Also, if the chip is the size of a wafer, is it appropriate to call it a Chip? Good question. I think it is. I mean the word "chip" isn't really that well defined (is HBM one chip?), but given that they sell it as a single unit and you can't really cut it in half I think it's one chip.

Looking at the picture, the thing is a platter. Not a chip. It's a really cool picture too.

It's a wafer, but that doesn't preclude it from being a chip. Where's you definition of chip coming from.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#148

Post by markonen »

sillysaurusx wrote: post #129
docfort wrote: post #127

Earlier quoted context omitted.

You might be interested in this for your M1 MBA: https://github.com/apple/tensorflow_macos

That was actually what I was referring to! It's super weird. I like it, and it's what I use primarily, but only because no other version of tensorflow will install. Their compiler is closed source, so it's almost impossible to tell what's going on. But, when I connect using tf.Session(), then .list_devices() shows only CPU available. However, when I enable the TF_MLC_LOGGING=1 magic variable, it does seem to be print…

With the disclaimer that I know nothing about this: doesn't the M1 have a separate "Neural Engine" for this? So it's using neither the CPU nor the GPU cores for TensorFlow?

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#149

Post by londons_explore »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

> That means you’re looking at two months of R&D minimum to get everything rewritten,

For a hobbiest, sure, that's a problem.

But for a big company with a big ML research team already, that isn't an issue - they just assign a few people to work on it, and in a few months it's done. If you're running any model at scale anyway you probably want to rewrite everything to make it run efficiently on your hardware.

Post reply on HN