Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’s giant chip will smash deep learning’s speed barrier

spectrum.ieee.org

Post reply on HN
Print view View original

51–60 of 108 posts

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#51

Post by tedivm »

wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

I got a demo of this two years ago, and honestly I don't think it matters that they aren't sharing these numbers. Any company that is going to consider this is going to want to benchmark it on their own models and systems, and as long as Cerebras allows that they aren't going to have trouble finding customers (assuming their claims line up with reality).

Even if that doesn't work out most of the people on these time have built companies that were acquired by either AMD or another chip maker.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#52

Post by rsp1984 »

This fits perfectly into the narrative of yesterday's discussion on HN [1].

Deep Neural Nets are somewhat of a brute force approach to machine learning. Training efficiency is horrible as compared with other ML approaches, but hey, as long as we can trade +5% of classification performance for +500% of NN complexity and throw more money at the problem, who cares?

I see a dystopian future where much better and much more efficient approaches to ML exist, but nobody's paying attention because we have Deep Neural Nets in hardware and decades of infrastructure supporting it.

[1] https://news.ycombinator.com/item?id=21929709

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#53

Post by corporate_shi11 »

mark_l_watson wrote: post #36

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much. I tend to be optimistic, so take my prediction with a grain o…

It's also my impression - from my modest exposure to DL over the past two years as a student taking courses - that deep learning must be overcome to reach AGI.

Specifically gradient descent is a post hoc approach to network tuning, while human neural connections are reinforced simultaneously as they fire together. The post hoc approach restricts the scope of the latent representations a network learns because such representations must serve a specific purpose (descending the gradient), while the human mind works by generating representations spontaneously at multiple levels of abstraction without any specific or immediate purpose in mind.

I believe the brain's ability to spontaneously generate latent representations capable of interacting with one another in a shared latent space is functionally enabled by the paradigm of neurons 'firing and wiring' together. I also believe it is the brain's ability to spontaneously generate hierarchically abstract representations in a shared space that is the key to AGI. We must therefore move away from gradient descent.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#54

Post by dnautics »

ThrowawayR2 wrote: post #16
green-eclipse wrote: post #14

The Cerebras chip really stands out in terms of the chip industry's relationship to Moore's law. Look at the graphs in this article for reference: https://medium.com/predict/cerebras-trounces-moores-law-with...

That article is utter balderdash. Yes, it's obvious that you can fit more transistors on a "chip" if you make the chip be much, much larger than what we ordinarily think of as a chip. No, it does not mean that Moore's Law has been invalidated or some new "AI Moore’s Law" (quoting from the post) has come into being.

> Yes, it's obvious that you can fit more transistors on a "chip" if you make the chip be much, much larger than what we ordinarily think of as a chip.

Without defending the article, it is however the case that simply scaling a chip size has nontrivial problems. For example, Will the piece of silicon warp or shatter if one side happens to get hotter than the other?

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#55

Post by luma »

tedivm wrote: post #48
luma wrote: post #17

Earlier quoted context omitted.

In most cases the cooling capacity is in fact the actual limit you are running up against. Getting more power into a rack is a simple matter of running more cable. Getting more power _out_ of the rack is a much more complicated issue to resolve.

I think it's a little more complicated than running more cables. Most datacenters have a total capacity they can handle, based on how many connections they have to their local grid (or grids, as datacenter places like Santa Clara have multiple power grids to give datacenter redundancy). You need to make sure your internal power distribution systems can actually handle the amount you want to push through, and you need…

Getting more power into a datacenter is a different problem than getting more (already available) power into a rack. I suppose I could have added "if your existing power distribution system can handle the extra power capacity". That includes service entrance, transfer switching, standby and backup power sources, and distribution to the rack level.

The point I'm trying to make is that, all things being equal, it's _much_ easier to handle un-equal power load between individual racks than it is to deal with the cooling side of the equation. Adding more power to a single rack usually just means a few more whips from your distribution. Getting that one extra-hot rack in the aisle to be effectively cooled requires a lot more infrastructure than running some cables.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#56

Post by innagadadavida »

mark_l_watson wrote: post #36

I don’t know if this mega-chip will be successful, but I like the idea. Before I retired I managed a deep learning team that had a very cool internal product for running distributed TensorFlow. Now in retirement I get by with a single 1070 GPU for experiments - not bad but having something much cheaper, much more memory, and much faster would help so much. I tend to be optimistic, so take my prediction with a grain o…

> I have to wonder if the market for super fast hardware for training models might match the smartphone market.

Recently Intel acquired Habana Labs for $2B [1] and Intel could possibly integrate this into upcoming CPUs (Intel sells more CPUs than iPhone for sure). However, this was on the inference side - unlike Cerebras which is making training faster. The most likely products to benefit from this would be Azure or AWS.

1. https://newsroom.intel.com/news-releases/intel-ai-acquisitio...

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#57

Post by IshKebab »

Veedrac wrote: post #38
recursivecaveat wrote: post #31

Earlier quoted context omitted.

As someone who works for another startup in this area, building the chip is only half the battle. The other half is tooling for compiling benchmark networks onto the chip in a performant manner. With 400k cores and their 'duplicate and re-route' defect strategy, this might literally be the most challenging compilation target ever made. It probably stacks up absolutely terribly in every metric right now. That's not to…

A bit baffled by this because on every axis I look this seems like a dream of a compilation target. * No DRAM or caches, everything is in SRAM, and all local SRAM loads are 1 cycle. * Model parallel alone is full performance, no need for data parallel if you size to fit. * Defects are handled in hardware; any latency differences are hidden & not in load path anyway. * Fully asynchronous/dataflow by default, only need…

I'm afraid recursivecaveat is right. This is an insanely difficult compilation target. I think you're possibly talking about a different kind of "compilation" - i.e. the Clang/GCC bit that converts C++ to machine code. That is indeed trivial. But "compilation" for these chips includes much more than that.

The really complicated bit is converting the tensorflow model to some kind of computation plan. Where do you put all the tensor data? How do you move it around the chip. It's insanely complicated. If anything kills Cerebras it will be the software.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#58

Post by justicezyx »

tedivm wrote: post #51
wbhart wrote: post #20

From the perspective of an outsider, I can't see how a company like this could survive. They claim on the one hand to have done something really amazing and are at the stage where they are looking for customers. Normally, you'd expect them to be touting performance figures to secure such investment. Instead, they've decided to keep the performance secret. And they've managed to find some "expert" who says this is nor…

I got a demo of this two years ago, and honestly I don't think it matters that they aren't sharing these numbers. Any company that is going to consider this is going to want to benchmark it on their own models and systems, and as long as Cerebras allows that they aren't going to have trouble finding customers (assuming their claims line up with reality). Even if that doesn't work out most of the people on these time…

Mass market customers are just going to skip without benchmarks.

Although, at this stage, Crebras does not care about mass market yet.

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#59

Post by justicezyx »

rsp1984 wrote: post #52

This fits perfectly into the narrative of yesterday's discussion on HN [1]. Deep Neural Nets are somewhat of a brute force approach to machine learning. Training efficiency is horrible as compared with other ML approaches, but hey, as long as we can trade +5% of classification performance for +500% of NN complexity and throw more money at the problem, who cares? I see a dystopian future where much better and much mor…

Well, if a better algorithm cannot beat DNN in a realistic product setting, then how can you say its better after all?

If the algorithm is indeed better, how can DNN dominates and turn into a dystonia...

Re: Cerebras’s giant chip will smash deep learning’s speed barrier

#60

Post by Veedrac »

IshKebab wrote: post #57
Veedrac wrote: post #38

Earlier quoted context omitted.

A bit baffled by this because on every axis I look this seems like a dream of a compilation target. * No DRAM or caches, everything is in SRAM, and all local SRAM loads are 1 cycle. * Model parallel alone is full performance, no need for data parallel if you size to fit. * Defects are handled in hardware; any latency differences are hidden & not in load path anyway. * Fully asynchronous/dataflow by default, only need…

I'm afraid recursivecaveat is right. This is an insanely difficult compilation target. I think you're possibly talking about a different kind of "compilation" - i.e. the Clang/GCC bit that converts C++ to machine code. That is indeed trivial. But "compilation" for these chips includes much more than that. The really complicated bit is converting the tensorflow model to some kind of computation plan. Where do you put…

It's model parallel, so the first thing you do is lay out your floorplan for the model, which looks like this.

https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo...

Then you put your data next to the core that uses it. Simples.

(Optimal placement is tricky, but approximate techniques work fine.)

Post reply on HN