Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’s giant chip will smash deep learning’s speed barrier

spectrum.ieee.org

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#71

Post by foota »

varelse wrote: post #68

I am far more excited by the underlying Wafer Scale Integration moonshot than I am by any AI benchmarks here. I know it's trendy to think there can only be one w/r to the AI Iron Throne but nope, not the case, everyone is writing bespoke code in production where the money is made. Well, almost everyone, Amazon seems to be the odd duck but they're a bunch of cheapskate thought leaders anyway (except for their offers t…

Isn't that similar to what AMD is doing with infinity fabric? Obviously not at such a large scale.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#72

Post by Veedrac »

jcranmer wrote: post #70
Veedrac wrote: post #38

Earlier quoted context omitted.

A bit baffled by this because on every axis I look this seems like a dream of a compilation target. * No DRAM or caches, everything is in SRAM, and all local SRAM loads are 1 cycle. * Model parallel alone is full performance, no need for data parallel if you size to fit. * Defects are handled in hardware; any latency differences are hidden & not in load path anyway. * Fully asynchronous/dataflow by default, only need…

Having worked on compilers for pretty weird architectures, it's generally the case that the less like a regular CPU your architecture is, the more difficult it is to compile. In particular, when you change the system from having to worry about how to optimally schedule a single state machine to having to place operations on a fixed routing grid (à la FPGA), the problem becomes radically different, and any looping con…

Remember that you aren't compiling arbitrary programs. Neural nets don't really have any local looping control flow, in the sense that data goes in one end and comes out the other. You'll have large-scale loops over the whole network, and each core might have a loop over small, local arrays of data, but you shouldn't have any sort of internal looping involving different parts of the model.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#73

Post by LASR »

varelse wrote: post #68

I am far more excited by the underlying Wafer Scale Integration moonshot than I am by any AI benchmarks here. I know it's trendy to think there can only be one w/r to the AI Iron Throne but nope, not the case, everyone is writing bespoke code in production where the money is made. Well, almost everyone, Amazon seems to be the odd duck but they're a bunch of cheapskate thought leaders anyway (except for their offers t…

This is pretty much on the same level as a raw material change that happened when CPUs went from a bunch of DIP ICs connected together to a single silicon die. LSI was a big leap in optimizing compute/$.

I had the same “hold the phone” reaction when I reading about AI but then saw the whole wafer in the marketing photo.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#74

Post by typon »

Veedrac wrote: post #32
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

They have customers already, one (Argonne National Labs) is given explicitly. The issue with using ‘industry standard’ benchmarks is that it's like measuring a bus' efficiency by shuttling around a single person at a time. The CS-1 is just bigger than that; the workloads that it provides the most value on are ones that are sized to fit, and specifically built for the device. This does make it hard to evaluate as outs…

They could always release figures for larger networks - they don't have to target Resnet50 (which is the MLPerf standard). I don't think anyone would hold it against them if they show massive improvements in something like GPT-2 training time (a network 37000x the size of Resnet)

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#75

Post by sanxiyn »

innagadadavida wrote: post #56
mark_l_watson wrote: post #36

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much. I tend to be optimistic, so take my prediction with a grain o…

> I have to wonder if the market for super fast hardware for training models might match the smartphone market. Recently Intel acquired Habana Labs for $2B [1] and Intel could possibly integrate this into upcoming CPUs (Intel sells more CPUs than iPhone for sure). However, this was on the inference side - unlike Cerebras which is making training faster. The most likely products to benefit from this would be Azure or…

Habana does both training and inference. Gaudi is for training, Goya is for inference.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#76

Post by 01100011 »

varelse wrote: post #68

I am far more excited by the underlying Wafer Scale Integration moonshot than I am by any AI benchmarks here. I know it's trendy to think there can only be one w/r to the AI Iron Throne but nope, not the case, everyone is writing bespoke code in production where the money is made. Well, almost everyone, Amazon seems to be the odd duck but they're a bunch of cheapskate thought leaders anyway (except for their offers t…

I just wonder about the reliability of a system that large. Sure, it's mostly used for machine learning where we don't seem to care as much, but what is the average MTBF of a chip this large? How many chips actually make it out of production?

Also, is this something that will likely scale up, or will this style of design hit a wall(power dissipation?) faster than, say, silicon-interconnect fabric?

Time will tell if this is the new path forward or just a curious footnote in the history of semiconductors.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#77

Post by 01100011 »

mark_l_watson wrote: post #36

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much. I tend to be optimistic, so take my prediction with a grain o…

Artificial 'neurons' used in deep learning networks are absolutely worlds apart from real biological neurons. I don't think anyone in the field seriously believes we will get to AGI via DL or our current models.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#78

Post by Google234 »

bcatanzaro wrote: post #5

Reminds me of that other great prediction of a GPU killer from IEEE Spectrum back in 2009: https://spectrum.ieee.org/computing/software/winner-multicor...

What did go wrong with Intel's MIC (Xeon Phi) project? I can't find a compressive account of this from HPC people. The idea seemed pretty sound: large die, simpler circuit, and much more parallelism, in the x86 line..

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#79

Post by 01100011 »

wolf550e wrote: post #61
Zenst wrote: post #2

A chip that size, imagine the yield. Equally, cooling - has to be water based as a heatsink that size would be on par to a small anvil and the weight factor would be some serious issues. Though unsure as no pictures of it in-play alas and all they say is - "20 kilowatts being consumed by each blew out into the Silicon Valley streets through a hole cut into the wall", which does somewhat beg for a picture as just rais…

Not 100% of the chip is enabled, they disable defective parts and don't advertise a model that has 100% parts enabled, so they don't need magical zero defect wafers. Images of the whole computer were published, you can see the massive cooling system: https://www.tomshardware.com/news/worlds-largest-chip-gets-a...

Did they come up with an architecture which can route around any defect? Probably not. Now, granted, 90% of their chip is probably dedicated to compute, but I'd bet there's some management infrastructure where they absolutely cannot tolerate a defect.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#80

Post by kingosticks »

01100011 wrote: post #79
wolf550e wrote: post #61

Earlier quoted context omitted.

Not 100% of the chip is enabled, they disable defective parts and don't advertise a model that has 100% parts enabled, so they don't need magical zero defect wafers. Images of the whole computer were published, you can see the massive cooling system: https://www.tomshardware.com/news/worlds-largest-chip-gets-a...

Did they come up with an architecture which can route around any defect? Probably not. Now, granted, 90% of their chip is probably dedicated to compute, but I'd bet there's some management infrastructure where they absolutely cannot tolerate a defect.

They'll simply have redundant copies of that logic. And they'll be physically located at areas of the wafer that yield well - some areas are much worse than others and I would imagine they'll make use of that.
Post reply on HN