Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Looming Battle Over AI Chips

barrons.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: The Looming Battle Over AI Chips

#31

Post by osteele »

Nokinside wrote: post #9

Nvidia will almost certainly respond to this challenge with it's own specialized machine learning and inference chips. It's probably what Google, Facebook and others hope. Forcing Nvidia to work harder is enough for them. Developing a new high performance microarchitecture for GPU or CPU is complex task. A new clean sheet design architecture takes 5-7 years even for teams that have been doing it constantly for decade…

Google’s gen2 TPU was announced May 2017, and available in beta February 2018. That 2018.02 date is probably the appropriate comparison to Volta’s 2017.12 and Zen’s 2017 dates.

EDIT: I’m trying to draw a comparison between the availability dates (and where the companies are now), not the start of production (and their development velocity). Including the announcement date was probably a red herring.

Re: The Looming Battle Over AI Chips

#32

Post by emcq »

There is certainly a lot of hype around AI chips, but I'm very skeptical of the reward. There are several technical concerns I have with any "AI" chip that ultimately leave you with something more general purpose (and not really an "AI" chip, but good at low precision matmul):

* For inference, how do you efficiently move your data to the chip? In general most of the time is spent in matmul, and there are lots of exciting DSPs, mobile GPUs, etc. that require a fair amount of jumping through hoops to get your data to the ML coprocessor. If you're doing anything low latency, good luck because you need tight control (or bypassing entirely) of the OS. Will this lead to a battle between chip makers? Seems more likely to be a battle between end to end platforms.

* For training, do you have an efficient data flow with distributed compute? For the foreseeable future any large model (or small model with lots of data) needs to be distributed. The bottlenecks that come from this limit the improvements from your new specialized architecture without good distributed computing. Again better chips don't really solve this, and comes from a platform. I've noticed many training loops have terrible GPU utilization, particularly with Tensorflow and V100s. Why does this happen? The GPU is so fast, but things like summary ops add to CPU time limiting perf. Bad data pipelines not actually pipelining transformations. Slow disks bottlenecking transfers. Not staging/pipelining transfers to the GPU. And then there is a bit of an open question of how to best pipeline transfers from the GPU. Is there a simulator feeding data? Then you have a whole new can of worms to train fast.

* For your chip architecture, do you have the right abstractions to train the next architecture efficiently? Backprop trains some wonderful nets but for the cost of a new chip (50-100M), and the time it takes to build (18 months min), how confident are you that the chip will still be relevant to the needs of your teams? This generally points you towards something more general purpose, which may leave some efficiency on the table. Eventually you end up at a low precision matmul core, which is the same thing everyone is moving towards or already doing whether you call yourself a GPU, DSP, or TPU (which is quite similar to DSPs).

Coming from an HPC/Graphics turned deep learning engineer, I've worked with gpus since 2006 and neural net chips since 2010 (before even AlexNet!!), so I'm a bit of an outlier here having seen so many perspectives. From my point of view the computational fabric exists we're just not using it well :)

Re: The Looming Battle Over AI Chips

#33

Post by Nokinside »

osteele wrote: post #31
Nokinside wrote: post #9

Nvidia will almost certainly respond to this challenge with it's own specialized machine learning and inference chips. It's probably what Google, Facebook and others hope. Forcing Nvidia to work harder is enough for them. Developing a new high performance microarchitecture for GPU or CPU is complex task. A new clean sheet design architecture takes 5-7 years even for teams that have been doing it constantly for decade…

Google’s gen2 TPU was announced May 2017, and available in beta February 2018. That 2018.02 date is probably the appropriate comparison to Volta’s 2017.12 and Zen’s 2017 dates. EDIT: I’m trying to draw a comparison between the availability dates (and where the companies are now), not the start of production (and their development velocity). Including the announcement date was probably a red herring.

I'm aware.

Making a chip and making competitive chip are two different things.

When Nvidia enters the market with specialized chip it's likely on completely another level in bandwidth, energy consumption and price per flop performance. They have so much more experience with this.

* https://drive.google.com/file/d/0Bx4hafXDDq2EMzRNcy1vSUxtcEk...

* https://blogs.nvidia.com/blog/2017/04/10/ai-drives-rise-acce...

Re: The Looming Battle Over AI Chips

#34

Post by _delirium »

bogomipz wrote: post #27
trimbo wrote: post #13

Earlier quoted context omitted.

Incognito -> search for headline -> click

Thank you for this tip. Out of curiosity why does this trick work?

Websites like this want traffic from Google. To get indexed by Googlebot they have to show the bot the article text, and Google's anti-blackhat-SEO rules mean that you have to show a human clicking through from Google the same text that you show Googlebot. So they have to show people visiting through that route the article text too.

Re: The Looming Battle Over AI Chips

#35

Post by justicezyx »

emcq wrote: post #32

There is certainly a lot of hype around AI chips, but I'm very skeptical of the reward. There are several technical concerns I have with any "AI" chip that ultimately leave you with something more general purpose (and not really an "AI" chip, but good at low precision matmul): * For inference, how do you efficiently move your data to the chip? In general most of the time is spent in matmul, and there are lots of exci…

Most top tier tech all.have their working solutions for these. It's a matter of turning into product and moving the industry mindset.

Re: The Looming Battle Over AI Chips

#36

Post by zantana »

deepnotderp wrote: post #18

Do people think that nobody at nVidia has ever heard of specialized deep learning processors? 1. Volta GPUs already have little matmul cores, basically a bunch of little TPUs. 2. The graphics dedicated silicon is an extremely tiny portion of the die, a trivial component (source: Bill Dally, nVidia chief scientist). 3. Memory access power and performance is the bottleneck (even in the TPU paper), and will only continu…

Is there an easy way that Nvidia can cripple their graphics targeted cards so they can't be used for GPCPU?

I'm thinking back to strategies like the 486SX https://en.wikipedia.org/wiki/Intel_80486SX

Re: The Looming Battle Over AI Chips

#37

Post by maffydub »

bogomipz wrote: post #17

The article states: >"LeCun and other scholars of machine learning know that if you were starting with a blank sheet of paper, an Nvidia GPU would not be the ideal chip to build. Because of the way machine-learning algorithms work, they are bumping up against limitations in the way a GPU is designed. GPUs can actually degrade the machine learning’s neural network, LeCun observed. “The solution is a different architec…

I went to a talk from the CTO of Graphcore (https://www.graphcore.ai/) on Monday. They are designing chips targeted at machine learning. As I understood it, their architecture comprises - lots of "tiles" - small processing cores with collocated memory (essentially DSPs) - very high bandwidth (90TB/s!) switching fabric to move data between tiles - "Bulk Synchronous Parallel" operation, meaning that the tiles do their work and then the switching fabric moves the data, and then we repeat.

The key challenge he pointed to was power - both in terms of getting energy in (modern CPUs/GPUs take similar current to your car starter motor!) and also getting the heat out. Logic gates take a lot more power than RAM, so he argued that collocating small chunks of RAM right next to your processing core was much better from a power perspective (meaning you could then pack yet more into your chip) as well as obviously being better from a performance perspective.

https://www.youtube.com/watch?v=Gh-Tff7DdzU isn't quite the presentation I saw, but it has quite a lot of overlap.

Hope that helps!

Re: The Looming Battle Over AI Chips

#38

Post by [deleted] »

lostgame wrote: post #19
oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

FYI I think your comment is informative and I understood a lot of it but that's a shitton of acronymns for the uninitiated.

[deleted]

Re: The Looming Battle Over AI Chips

#39

Post by brandmeyer »

osteele wrote: post #31
Nokinside wrote: post #9

Nvidia will almost certainly respond to this challenge with it's own specialized machine learning and inference chips. It's probably what Google, Facebook and others hope. Forcing Nvidia to work harder is enough for them. Developing a new high performance microarchitecture for GPU or CPU is complex task. A new clean sheet design architecture takes 5-7 years even for teams that have been doing it constantly for decade…

Google’s gen2 TPU was announced May 2017, and available in beta February 2018. That 2018.02 date is probably the appropriate comparison to Volta’s 2017.12 and Zen’s 2017 dates. EDIT: I’m trying to draw a comparison between the availability dates (and where the companies are now), not the start of production (and their development velocity). Including the announcement date was probably a red herring.

Announcing it to the world != start of design effort.

Re: The Looming Battle Over AI Chips

#40

Post by arca_vorago »

oneshot908 wrote: post #15
joe_the_user wrote: post #12

Nvidia, moreover, increasingly views its software for programming its chips, called CUDA, as a kind of vast operating system that would span all of the machine learning in the world, an operating system akin to what Microsoft (MSFT) was in the old days of PCs. Yeah, nVidia throwing it's weight around in terms of requiring that data centers pay more to use cheap consumer gaming chips may turn out to backfire and certa…

It probably won't. For every oppressive move NVDA has made so far, there has been a swarm of low-information technophobe MBA sorts who eat their computational agitprop right up, some of them even fashion themselves as data scientists. More likely, NVDA continues becoming the Oracle of AI that everyone needs and everyone hates.

So is OpenCL dead? Because that's how everyone is talking. The tools you choose, and their licensing, matters!
Post reply on HN