Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras’ new monster AI chip adds 1.4T transistors

spectrum.ieee.org

Post reply on HN
Print view View original

101–110 of 169 posts

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#101

Post by oivey »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

Graphics APIs and deep learning frameworks are really nothing alike. In graphics, save for a few new features, the goal every year is the same: draw more and faster. Deep learning is evolving much faster than that. The answer isn’t an API; it’s powerful and mostly transparent automatic differentiation. Anything that is just an API will forever be behind and is much less useful for active research. There are a few projects around improving AD support, but it’s a relatively difficult problem because to do it best you need deep compiler integration.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#102

Post by sillysaurusx »

systemvoltage wrote: post #98
sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

You're bearish on new hardware for AI training because it takes 2 months to setup? That's like saying you're bearish on the invention of transistor because there are no computers yet. This chip is for research organizations such as LLNL and ANL. Tooling and commoditization will happen. > “if we build it, they’ll buy it.” This is literally how any new thing is invented and commercialized.

Well, two months has a way of turning into two years. June 2019 was right around when I fell in love with TPUs, and we’re still happily TPUing today.

Sometimes, though, it would be nice to simply focus on doing interesting ML work rather than wrestling with the intricacies of tensor slicing and memory access pattern optimization (to say nothing of the cursed inability to manually manage memory, resulting in explosions of allocations in the forward pass for unclear reasons).

Let’s put it this way. If you contract me to implement a full gpt model on this new hardware, two months full time work would be my minimum estimate. That’s 40 hours a week of focused effort, with no breaks and no other projects. I don’t know about you, but two months with nothing to show can demoralize most teams I’ve worked with.

The point was, we’re early in ML’s lifecycle. And, like React for webdev, people are slow to change their habits — for better or worse, pytorch and tensorflow are the APIs people think in. So if you want your hardware to be widely adopted, you need tooling that supports the workflows people have spent months learning.

Jax is on the horizon too. Supposedly they’re launching something soon that might tip the scale in their favor. Perhaps an ambitious hardware vendor could capture the future market by preemptively implementing Jax support. But perhaps not: at that point you’d be competing toe to toe with Google’s TPU offering, since Jax is Google’s dogfood.

Right now my money is on TPUs, partly because of their fantastic support staff. But maybe some other company will come along and offer a better integrated cloud experience.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#104

Post by justaguy88 »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

You might like this talk that describes parts of the compiler ecosystem that are lacking standardization for accelerators

https://www.youtube.com/watch?v=4HgShra-KnY

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#106

Post by durst »

sillysaurusx wrote: post #102
systemvoltage wrote: post #98

Earlier quoted context omitted.

You're bearish on new hardware for AI training because it takes 2 months to setup? That's like saying you're bearish on the invention of transistor because there are no computers yet. This chip is for research organizations such as LLNL and ANL. Tooling and commoditization will happen. > “if we build it, they’ll buy it.” This is literally how any new thing is invented and commercialized.

Well, two months has a way of turning into two years. June 2019 was right around when I fell in love with TPUs, and we’re still happily TPUing today. Sometimes, though, it would be nice to simply focus on doing interesting ML work rather than wrestling with the intricacies of tensor slicing and memory access pattern optimization (to say nothing of the cursed inability to manually manage memory, resulting in explosion…

Do you have a longer writeup on your experience performance tuning for the TPU? Is https://cloud.google.com/tpu/docs/performance-guide the best resource in this area?

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#107

Post by frostburg »

arisAlexis wrote: post #62
jefft255 wrote: post #50

Earlier quoted context omitted.

It's because you made a false analogy. AI isn't literally "the future". Billions of $ are being invested in deep-learning focused AI right now (which you call "the future"), and yes it could be a bubble and it could burst. You can disagree, but it's still a sensible thing to predict.

By bursting you mean humanity will never create artificial intelligence? Or you mean that there will be a cool off period as for example what happened with quantum physics at some point? Because it sure looks to me that there is no future without AI regardless of cool off periods. That makes my statement true. If you think humanity will never progress from where we are now then we pretty much are on very opposite sch…

The point being discussed here is that the current approaches (and those investing in them) might fail to yield any further significant advance.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#108

Post by sanxiyn »

sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

> What we need is a unified API that can easily support multiple backends

Android standardized NNAPI and on-device inference is in pretty good shape. As you said, training is different matter.

Re: Cerebras’ new monster AI chip adds 1.4T transistors

#110

Post by sanxiyn »

oivey wrote: post #101
sillysaurusx wrote: post #78

I’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and w…

Graphics APIs and deep learning frameworks are really nothing alike. In graphics, save for a few new features, the goal every year is the same: draw more and faster. Deep learning is evolving much faster than that. The answer isn’t an API; it’s powerful and mostly transparent automatic differentiation. Anything that is just an API will forever be behind and is much less useful for active research. There are a few pro…

The answer is an API, like NNAPI. AD is a frontend concern and doesn't really matter to accelerators.

For AD, I am bullish for Enzyme, which does AD on LLVM IR, avoiding deep compiler integration: https://enzyme.mit.edu/

Post reply on HN