Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’s giant chip will smash deep learning’s speed barrier

spectrum.ieee.org

Post reply on HN
Print view View original

61–70 of 108 posts

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#61

Post by wolf550e »

Zenst wrote: post #2

A chip that size, imagine the yield. Equally, cooling - has to be water based as a heatsink that size would be on par to a small anvil and the weight factor would be some serious issues. Though unsure as no pictures of it in-play alas and all they say is - "20 kilowatts being consumed by each blew out into the Silicon Valley streets through a hole cut into the wall", which does somewhat beg for a picture as just rais…

Not 100% of the chip is enabled, they disable defective parts and don't advertise a model that has 100% parts enabled, so they don't need magical zero defect wafers.

Images of the whole computer were published, you can see the massive cooling system: https://www.tomshardware.com/news/worlds-largest-chip-gets-a...

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#62

Post by GaryNumanVevo »

mark_l_watson wrote: post #36

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much. I tend to be optimistic, so take my prediction with a grain o…

Cerebras is a reaction to the recent Deep Learning trend. Larger networks, supposedly better performance. As someone doing distributed training regularly, I've seen some super inefficient models that take 3x more resources (time / compute / bandwidth) for a 2% bump. I think we'll see a big wave in NN optimization in the near future.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#64

Post by someguyorother »

justicezyx wrote: post #59
rsp1984 wrote: post #52

This fits perfectly into the narrative of yesterday's discussion on HN [1]. Deep Neural Nets are somewhat of a brute force approach to machine learning. Training efficiency is horrible as compared with other ML approaches, but hey, as long as we can trade +5% of classification performance for +500% of NN complexity and throw more money at the problem, who cares? I see a dystopian future where much better and much mor…

Well, if a better algorithm cannot beat DNN in a realistic product setting, then how can you say its better after all? If the algorithm is indeed better, how can DNN dominates and turn into a dystonia...

What economists call path dependence.

The alternative algorithm would be better than DNN if the same amount of effort was put into creating special-purpose hardware, libraries, and so on; but in the dystopia, it's not fully refined DNN vs fully refined alternative algorithm, but fully refined DNN vs alternative algorithm with hardware and software optimized for DNN.

The alternative algorithm always looks unappealing because the playing field historically favors DNN, and so doesn't take off in the dystopia.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#65

Post by Veedrac »

atq2119 wrote: post #18
green-eclipse wrote: post #14

The Cerebras chip really stands out in terms of the chip industry's relationship to Moore's law. Look at the graphs in this article for reference: https://medium.com/predict/cerebras-trounces-moores-law-with...

That article is hogwash. Sure, the Cerebras "chip" is impressive. But the idea that it will accelerate Moore's law and usher in the singularity is just nonsense. Nobody has even made serious efforts to use deep learning for physical design, and its scope for improving designs is limited at best even in theory. If this was trying to aim at solid state physics and materials research, then maybe one could be carefully o…

> Nobody has even made serious efforts to use deep learning for physical design

DeepMind have for place&route IIRC.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#66

Post by baybal2 »

frankchn wrote: post #19
Zenst wrote: post #2

A chip that size, imagine the yield. Equally, cooling - has to be water based as a heatsink that size would be on par to a small anvil and the weight factor would be some serious issues. Though unsure as no pictures of it in-play alas and all they say is - "20 kilowatts being consumed by each blew out into the Silicon Valley streets through a hole cut into the wall", which does somewhat beg for a picture as just rais…

Chiplet designs means that you still have to route signals either onto an interposer or onto a PCB. If you have a silicon interposer you have the same issue of making a really large silicon die. If you route into the PCB, then you may need SerDes depending on what you do and bandwidth will be lower and latency will be higher due to signal integrity issues. Maybe something like Intel's EMIB technology where they have…

I will dissent. Organic interposers are dirt cheap, and nearly as good unless all you want is density.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#67

Post by baybal2 »

ZhuanXia wrote: post #63

Them shunning benchmarks is pretty lame.

The guy who runs Cerebas has history of quick selling companies that then went nowhere. He bets all on wow-effect, and sells to trend chasing suckers.

Less than stellar benchmarks will ruin the "magic"

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#68

Post by varelse »

I am far more excited by the underlying Wafer Scale Integration moonshot than I am by any AI benchmarks here. I know it's trendy to think there can only be one w/r to the AI Iron Throne but nope, not the case, everyone is writing bespoke code in production where the money is made. Well, almost everyone, Amazon seems to be the odd duck but they're a bunch of cheapskate thought leaders anyway (except for their offers to junior engineers in their desperate hail mary attempt to catch up with FAIR and DeepMind, but... I... digress...).

Which is to say that graphs written to run specifically on Cerebras's giant chip will smash deep learning's speed barrier for graphs written to run best on Cerebras's giant chip. And that's great, but it won't be every graph, there is no free lunch. Hear me now, believe me later(tm).

But if we can cut the cost of interconnect by putting a figurative datacenter's worth of processors on a chip, that's genuinely interesting, and it has applications far beyond the multiplies and adds of AI. But be very wary of anyone wielding the term "sparse" for it is a massively overloaded definition and every single one of those definitions is a beautiful and unique snowflake w/r to efficient execution on bespoke HW.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#69

Post by [deleted] »

Veedrac wrote: post #60
IshKebab wrote: post #57

Earlier quoted context omitted.

I'm afraid recursivecaveat is right. This is an insanely difficult compilation target. I think you're possibly talking about a different kind of "compilation" - i.e. the Clang/GCC bit that converts C++ to machine code. That is indeed trivial. But "compilation" for these chips includes much more than that. The really complicated bit is converting the tensorflow model to some kind of computation plan. Where do you put…

It's model parallel, so the first thing you do is lay out your floorplan for the model, which looks like this. https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo... Then you put your data next to the core that uses it. Simples. (Optimal placement is tricky, but approximate techniques work fine.)

[deleted]

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#70

Post by jcranmer »

Veedrac wrote: post #38
recursivecaveat wrote: post #31

Earlier quoted context omitted.

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

A bit baffled by this because on every axis I look this seems like a dream of a compilation target. * No DRAM or caches, everything is in SRAM, and all local SRAM loads are 1 cycle. * Model parallel alone is full performance, no need for data parallel if you size to fit. * Defects are handled in hardware; any latency differences are hidden & not in load path anyway. * Fully asynchronous/dataflow by default, only need…

Having worked on compilers for pretty weird architectures, it's generally the case that the less like a regular CPU your architecture is, the more difficult it is to compile.

In particular, when you change the system from having to worry about how to optimally schedule a single state machine to having to place operations on a fixed routing grid (à la FPGA), the problem becomes radically different, and any looping control flow becomes an absolute nail-biter of an issue.

Post reply on HN