Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’s giant chip will smash deep learning’s speed barrier

spectrum.ieee.org

Post reply on HN
Print view View original

41–50 of 108 posts

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#41

Post by ivalm »

Zenst wrote: post #2

A chip that size, imagine the yield. Equally, cooling - has to be water based as a heatsink that size would be on par to a small anvil and the weight factor would be some serious issues. Though unsure as no pictures of it in-play alas and all they say is - "20 kilowatts being consumed by each blew out into the Silicon Valley streets through a hole cut into the wall", which does somewhat beg for a picture as just rais…

> A chip that size, imagine the yield

From discussion at a demo the yield is good, since they are using a large node. Their hardware rerouting also mitigates defects on most chips.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#42

Post by mlyle »

atq2119 wrote: post #18
green-eclipse wrote: post #14

The Cerebras chip really stands out in terms of the chip industry's relationship to Moore's law. Look at the graphs in this article for reference: https://medium.com/predict/cerebras-trounces-moores-law-with...

That article is hogwash. Sure, the Cerebras "chip" is impressive. But the idea that it will accelerate Moore's law and usher in the singularity is just nonsense. Nobody has even made serious efforts to use deep learning for physical design, and its scope for improving designs is limited at best even in theory. If this was trying to aim at solid state physics and materials research, then maybe one could be carefully o…

Yah, it's BS. But it may be teaching TSMC a whole lot about making larger chips with good yield, and the across-reticle interconnect technology is impressive too and may find some general applicability (e.g. it sounds like something AMD might like).

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#43

Post by Accujack »

Zenst wrote: post #2

A chip that size, imagine the yield. Equally, cooling - has to be water based as a heatsink that size would be on par to a small anvil and the weight factor would be some serious issues. Though unsure as no pictures of it in-play alas and all they say is - "20 kilowatts being consumed by each blew out into the Silicon Valley streets through a hole cut into the wall", which does somewhat beg for a picture as just rais…

>which does somewhat beg for a picture as just raises more questions.

There's a picture in the article.

>Why would they make a chip this big

Did you read the article?

>this article mentions TSMC. Another article I read that they used a 16nm node

Yes, 16nm/TSMC.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#44

Post by atq2119 »

mlyle wrote: post #42
atq2119 wrote: post #18

Earlier quoted context omitted.

That article is hogwash. Sure, the Cerebras "chip" is impressive. But the idea that it will accelerate Moore's law and usher in the singularity is just nonsense. Nobody has even made serious efforts to use deep learning for physical design, and its scope for improving designs is limited at best even in theory. If this was trying to aim at solid state physics and materials research, then maybe one could be carefully o…

Yah, it's BS. But it may be teaching TSMC a whole lot about making larger chips with good yield, and the across-reticle interconnect technology is impressive too and may find some general applicability (e.g. it sounds like something AMD might like).

Oh, for sure there are things to be learned from this. The responsibility for yield doesn't lie with TSMC though, but with the logic design: to make this kind of integration work, your design has to be able to tolerate a fault essentially anywhere on the wafer surface.

This isn't magic, of course: keep in mind that we already have SRAM with extra capacity for fault tolerance, and multi-core chips which are binned based on the number of functioning cores has been standard for a long time.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#45

Post by mlyle »

atq2119 wrote: post #44
mlyle wrote: post #42

Earlier quoted context omitted.

Yah, it's BS. But it may be teaching TSMC a whole lot about making larger chips with good yield, and the across-reticle interconnect technology is impressive too and may find some general applicability (e.g. it sounds like something AMD might like).

Oh, for sure there are things to be learned from this. The responsibility for yield doesn't lie with TSMC though, but with the logic design: to make this kind of integration work, your design has to be able to tolerate a fault essentially anywhere on the wafer surface. This isn't magic, of course: keep in mind that we already have SRAM with extra capacity for fault tolerance, and multi-core chips which are binned bas…

Design to tolerate failures is only one variable in yields. Wafer-scale integration exercises to the limit both our ability to tolerate defects and our ability to minimize them.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#46

Post by cepp »

luma wrote: post #17
Zenst wrote: post #13

Earlier quoted context omitted.

Wow, that really does add some perspective upon the cooling and the aspect about power requirements datacenter wise really does highlight how out-there these type of systems are over the usual rack layouts. Equally, the cooling capacity of the datacenter comes into play with such systems. Given the power density, the amount of heat being generated would equally be above your normal rack output.

In most cases the cooling capacity is in fact the actual limit you are running up against. Getting more power into a rack is a simple matter of running more cable. Getting more power _out_ of the rack is a much more complicated issue to resolve.

I think this is also a paradigm problem. Modern chipset advancements are at the crossroads of power vs. cooling. The logical extension of that fight is greater power and cooling requirements in the DC which it is not necessarily equipped to provide by default.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#47

Post by michelpp »

The members of the GraphBLAS forum have discussed this chip a couple of times. There's a lot of research on making deep neural networks more sparse, not just by pruning a dense matrix, but by starting with a sparse matrix structure de novo. Lincoln Laboratory's Dr. Jeremy Kepner has a good paper on Radix-Net mixed radix topologies that achieve good learning ability but with far fewer neurons and memory requirements. Cited in the paper was a network constructed with these techniques that simulated the size and sparsity of the human brain:

https://arxiv.org/pdf/1905.00416.pdf

It would be cool to see the GraphBLAS API ported to this chip, which from what I can tell comes with sparse matrix processing units. As networks become bigger, deeper, but sparser, a chip like this will have some demonstrable advantages over dense numeric processors like GPUs.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#48

Post by tedivm »

luma wrote: post #17
Zenst wrote: post #13

Earlier quoted context omitted.

Wow, that really does add some perspective upon the cooling and the aspect about power requirements datacenter wise really does highlight how out-there these type of systems are over the usual rack layouts. Equally, the cooling capacity of the datacenter comes into play with such systems. Given the power density, the amount of heat being generated would equally be above your normal rack output.

In most cases the cooling capacity is in fact the actual limit you are running up against. Getting more power into a rack is a simple matter of running more cable. Getting more power _out_ of the rack is a much more complicated issue to resolve.

I think it's a little more complicated than running more cables. Most datacenters have a total capacity they can handle, based on how many connections they have to their local grid (or grids, as datacenter places like Santa Clara have multiple power grids to give datacenter redundancy). You need to make sure your internal power distribution systems can actually handle the amount you want to push through, and you need to ensure that your backup power is actually enough to get you through major outages.

AWS, as an example, tends to only have 20MW to 30MW for each of their datacenters- anything above that they say isn't worth the hassle when they can just open a new datacenter. Power is definitely a limiting factor.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#49

Post by HereBeBeasties »

dnautics wrote: post #35
recursivecaveat wrote: post #31

Earlier quoted context omitted.

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

Is it not the case that the defect identification and rerouting happens at the hw level in a QA phase post production? If not I'm even less bullish on cerebras.

Yes, that's what their web site says.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#50

Post by tedivm »

cepp wrote: post #46
luma wrote: post #17

Earlier quoted context omitted.

In most cases the cooling capacity is in fact the actual limit you are running up against. Getting more power into a rack is a simple matter of running more cable. Getting more power _out_ of the rack is a much more complicated issue to resolve.

I think this is also a paradigm problem. Modern chipset advancements are at the crossroads of power vs. cooling. The logical extension of that fight is greater power and cooling requirements in the DC which it is not necessarily equipped to provide by default.

This is why I thought what Colovore did was pretty smart- they built liquid cooling into all of their racks. They are literally the densest datacenter I've found that actually allows people to colo with them (I'm sure there's plenty of companies who own their own datacenters that might be denser), but even with their systems you'd only be able to fit two of the Cerebras systems in a single rack (and you wouldn't be able to power both up 100% at the same time).

https://www.colovore.com/data-center/

Post reply on HN