Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’s giant chip will smash deep learning’s speed barrier

spectrum.ieee.org

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#31

Post by recursivecaveat »

wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to say it will necessarily get better, most of the people I've talked to don't think the megachip will ultimately amount to much more than a clever marketing ploy.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#32

Post by Veedrac »

wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

They have customers already, one (Argonne National Labs) is given explicitly.

The issue with using ‘industry standard’ benchmarks is that it's like measuring a bus' efficiency by shuttling around a single person at a time. The CS-1 is just bigger than that; the workloads that it provides the most value on are ones that are sized to fit, and specifically built for the device.

This does make it hard to evaluate as outsiders (certainly for similar reasons I never liked Graphcore), but I don't think it means anything as grim as you say. The recipe fits.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#33

Post by privateSFacct »

jandrese wrote: post #26
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

Keeping the performance figures a secret is a red flag on the level of "run, don't walk, away from this company". At best their solution is on par with GPUs in a performance per watt/dollar sense. At worst they're scammers looking for a sucker.

I'm also very curious about their performance per watt / dollar for the standard ML datasets out there (facial recognition etc). We have a reasonable sense of both training time and runtime for these in the cloud (and it is falling FAST).

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#34

Post by m0zg »

Veedrac wrote: post #32
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

They have customers already, one (Argonne National Labs) is given explicitly. The issue with using ‘industry standard’ benchmarks is that it's like measuring a bus' efficiency by shuttling around a single person at a time. The CS-1 is just bigger than that; the workloads that it provides the most value on are ones that are sized to fit, and specifically built for the device. This does make it hard to evaluate as outs…

That' sounds like horseshit to me. Very large public datasets and models are available to test training on a chip or system of any size. ImageNet is large enough for this. But if that's not sufficient, OpenImages is also available.

To me as a practitioner a meaningful metric would be "it trains an ImageNet classifier to e.g. 80% top1 in a minute". If it's not suitable for CNNs, do BERT or something else non-convolutional. Even better if I can replicate this result in a public cloud somewhere. They know this, and yet all we have is a single mention of a customer under an NDA and no public benchmarks of any kind, let alone any verifiable ones. If it did excel at those, we'd already know.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#35

Post by dnautics »

recursivecaveat wrote: post #31
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

Is it not the case that the defect identification and rerouting happens at the hw level in a QA phase post production? If not I'm even less bullish on cerebras.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#36

Post by mark_l_watson »

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much.

I tend to be optimistic, so take my prediction with a grain of salt: I bet within 7 or 8 years there will be an inexpensive device that will blow away what we have now. There are so many applications for much larger end to end models that will but pressure on the market for something much better than what we have now. BTW, the ability to efficiently run models on my new iPhone 11 Pro is impressive and I have to wonder if the market for super fast hardware for training models might match the smartphone market. For this to happen, we need a deep learning rules the world shift. BTW, off topic, but I don’t think deep learning gets us to AGI.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#37

Post by joe_the_user »

recursivecaveat wrote: post #31
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made.

While I'd be generally skeptical, it seems like the compilation for the rerouting could be done on a single low level, below whatever their assembler is, and so the could just look like a regular array of cores - just a single array that translates from i to the ith "real" core and similar structures seems like it could be enough.

Edit: I mean, if they're smart, it seems like they'd make the thing look as much as possible like a generic GPU capable of OpenCL. I have no idea if they'll do that but since they have size, they won't have to sell their stuff an otherwise custom approach.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#38

Post by Veedrac »

recursivecaveat wrote: post #31
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

A bit baffled by this because on every axis I look this seems like a dream of a compilation target.

* No DRAM or caches, everything is in SRAM, and all local SRAM loads are 1 cycle.

* Model parallel alone is full performance, no need for data parallel if you size to fit.

* Defects are handled in hardware; any latency differences are hidden & not in load path anyway.

* Fully asynchronous/dataflow by default, only need minimal synchronization between forward/backward passes.

I genuinely don't know how you'd build a simpler system than this.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#39

Post by tynpeddler »

m0zg wrote: post #34
Veedrac wrote: post #32

Earlier quoted context omitted.

They have customers already, one (Argonne National Labs) is given explicitly. The issue with using ‘industry standard’ benchmarks is that it's like measuring a bus' efficiency by shuttling around a single person at a time. The CS-1 is just bigger than that; the workloads that it provides the most value on are ones that are sized to fit, and specifically built for the device. This does make it hard to evaluate as outs…

That' sounds like horseshit to me. Very large public datasets and models are available to test training on a chip or system of any size. ImageNet is large enough for this. But if that's not sufficient, OpenImages is also available. To me as a practitioner a meaningful metric would be "it trains an ImageNet classifier to e.g. 80% top1 in a minute". If it's not suitable for CNNs, do BERT or something else non-convoluti…

> Cerebras hasn’t released MLPerf results or any other independently verifiable apples-to-apples comparisons. Instead the company prefers to let customers try out the CS-1 using their own neural networks and data.

> This approach is not unusual, according to analysts. “Everybody runs their own models that they developed for their own business,” says Karl Freund, an AI analyst at Moor Insights. “That’s the only thing that matters to buyers.”

Sounds like instead of benchmarks, prospective customers get a chance to run a workload of their choice on the core before purchase. Assuming support is good, that's way better than looking at benchmarks, because you're guaranteed that the performances you're comparing are for workloads you care about.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#40

Post by Veedrac »

m0zg wrote: post #34
Veedrac wrote: post #32

Earlier quoted context omitted.

They have customers already, one (Argonne National Labs) is given explicitly. The issue with using ‘industry standard’ benchmarks is that it's like measuring a bus' efficiency by shuttling around a single person at a time. The CS-1 is just bigger than that; the workloads that it provides the most value on are ones that are sized to fit, and specifically built for the device. This does make it hard to evaluate as outs…

That' sounds like horseshit to me. Very large public datasets and models are available to test training on a chip or system of any size. ImageNet is large enough for this. But if that's not sufficient, OpenImages is also available. To me as a practitioner a meaningful metric would be "it trains an ImageNet classifier to e.g. 80% top1 in a minute". If it's not suitable for CNNs, do BERT or something else non-convoluti…

The appropriately large models with public recognition I know of use attention, which is too memory-hungry to work effectively on the CS-1. The datasets aren't the issue.

I'm fine with skepticism. It's certainly plausible that they don't actually do all that well.

Post reply on HN