Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Looming Battle Over AI Chips

barrons.com

Post reply on HN
Print view View original

11–20 of 89 posts

Re: The Looming Battle Over AI Chips

#11

Post by Symmetry »

etaioinshrdlu wrote: post #8

It would be interesting to try emulate a many-core CPU as a GPU program and then run an OS on it. This sounds like a dumb idea, and it probably is. But consider a few things: * NVIDIA GPUs have exceptional memory bandwidth, and memory can be a slow resource on CPU based systems (perhaps limited by latency more than bandwidth) * The clock speed isn't that slow, it's in the GHz. Still one's clocks per emulated instruct…

Generally speaking emulating special purpose hardware in software slows things down a lot so I don't think that relying on a software branch predictor is going to result in performance anywhere close to what you'd see in, say, an ARM A53. And since you have to trade off clock cycles used in your branch predictor with clock cycles in your main thread I think it would be a net loss. Remember that even though NVidia calls each execution port a "Core" it can only execute one instruction across all of them at a time. The advantage over regular SIMD is that each shader processor tracks its own PC and only executes the broadcast instruction if it's appropriate - allowing diverging control flows across functions in ways that normal SIMD+mask would have a very hard time with except in the lowest level of a compute kernel.

That also means that you can really only emulate as many cores as the NVidia card has streaming multiprocessors, not as many as it has shared processors or "cores".

Also, it's true that GPUs have huge memory bandwidth they achieve that by trading off against memory latency. You can actually think of GPUs as throughput optimized compute devices and CPUs as latency optimized compute devices and not be very mislead.

So I expect that the single threaded performance of a NVidia general purpose computer to be very low in cases where the memory and branch patterns aren't obvious enough to be predictable to the compiler. Not unusably slow but something like the original Raspberry Pi.

Each emulated core would certainly have very good SIMD support but at the same time pretending that they're just SIMD would sacrifice the extra flexibility that NVidia's SIMT model gives you.

Re: The Looming Battle Over AI Chips

#12

Post by joe_the_user »

Nvidia, moreover, increasingly views its software for programming its chips, called CUDA, as a kind of vast operating system that would span all of the machine learning in the world, an operating system akin to what Microsoft (MSFT) was in the old days of PCs.

Yeah, nVidia throwing it's weight around in terms of requiring that data centers pay more to use cheap consumer gaming chips may turn out to backfire and certainly has an abusive-monopoly flavor to it.

As I've researched the field, Cuda really seems provides considerable value to the individual programmer. But making maneuvers of this sort may show the limits of that sort of advantage.

https://www.theregister.co.uk/2018/01/03/nvidia_server_gpus/

Re: The Looming Battle Over AI Chips

#14

Post by oneshot908 »

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this.

Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA for Titan V such that going forward, you cannot deploy Titan V in a datacenter for anything but mining cryptocurrency, and his company reserves the right going forward to audit your use of their SW and HW at any time to force compliance with whatever rules Jensen pulled out of his butt that day after his morning weed. And that's a shame. Because there's no way any of these companies can beat the !/$ of consumer GPUs and NVDA is lying out of its a$$ to say you can't do HPC on them.

But beyond NVDA shenanigans, I think it's incredibly risky to second guess those siloed computation graphs from the outside in the hopes of anything but an acqui-hire for an internal effort. Things ended well for Nervana even if their HW didn't ship in time, but when I see a 2018 company (http://nearist.ai/k-nn-benchmarks-part-wikipedia) comparing their unavailable powerpoint processor to GPUs from 2013, and then doubling down on doing so when someone rightly points out how stupid that is, I see a beached fail whale in the making, not a threat to NVDA's Deepopoly.

Re: The Looming Battle Over AI Chips

#15

Post by oneshot908 »

joe_the_user wrote: post #12

Nvidia, moreover, increasingly views its software for programming its chips, called CUDA, as a kind of vast operating system that would span all of the machine learning in the world, an operating system akin to what Microsoft (MSFT) was in the old days of PCs. Yeah, nVidia throwing it's weight around in terms of requiring that data centers pay more to use cheap consumer gaming chips may turn out to backfire and certa…

It probably won't. For every oppressive move NVDA has made so far, there has been a swarm of low-information technophobe MBA sorts who eat their computational agitprop right up, some of them even fashion themselves as data scientists. More likely, NVDA continues becoming the Oracle of AI that everyone needs and everyone hates.

Re: The Looming Battle Over AI Chips

#16

Post by joe_the_user »

etaioinshrdlu wrote: post #8

It would be interesting to try emulate a many-core CPU as a GPU program and then run an OS on it. This sounds like a dumb idea, and it probably is. But consider a few things: * NVIDIA GPUs have exceptional memory bandwidth, and memory can be a slow resource on CPU based systems (perhaps limited by latency more than bandwidth) * The clock speed isn't that slow, it's in the GHz. Still one's clocks per emulated instruct…

Support for specialized CPU functions won't happen and doesn't make sense.

However, it is quite feasible to emulate, on a GPU, a networked group of a general purpose CPUs( ie, run MIMD[1] programs on SIMD[2] architecture). This, MOG[3], has been a project of Henry G. Dietz of the University of Kentucky. Unfortunately, the project seems to have stalled at a "rough" level. He claims that he can run MIMD programs at 1/4 efficiency while also running SIMD programs at near full efficiency. His video is instructive [4].

Edit: Note that this isn't intended for deep learning applications as such but rather for traditional supercomputing applications (weather prediction, other physics simulations ,etc).

[1] https://en.wikipedia.org/wiki/MIMD [2] https://en.wikipedia.org/wiki/SIMD [3] http://aggregate.org/MOG/ [4] https://www.youtube.com/watch?v=FZ6efZFlzRQ

Re: The Looming Battle Over AI Chips

#17

Post by bogomipz »

The article states:

>"LeCun and other scholars of machine learning know that if you were starting with a blank sheet of paper, an Nvidia GPU would not be the ideal chip to build. Because of the way machine-learning algorithms work, they are bumping up against limitations in the way a GPU is designed. GPUs can actually degrade the machine learning’s neural network, LeCun observed.

“The solution is a different architecture, one more specialized for neural networks,” said LeCun."

Could someone explain to me what exactly are the limitations of current GPGUs such as those sold by Nvidia when used in machine learning/AI contexts? Are these limitation only experienced at scale? Ff someone has resources or links they could share regarding these limitations and better designs I would greatly appreciate it.

Re: The Looming Battle Over AI Chips

#18

Post by deepnotderp »

Do people think that nobody at nVidia has ever heard of specialized deep learning processors?

1. Volta GPUs already have little matmul cores, basically a bunch of little TPUs.

2. The graphics dedicated silicon is an extremely tiny portion of the die, a trivial component (source: Bill Dally, nVidia chief scientist).

3. Memory access power and performance is the bottleneck (even in the TPU paper), and will only continue to get worse.

Re: The Looming Battle Over AI Chips

#19

Post by lostgame »

oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

FYI I think your comment is informative and I understood a lot of it but that's a shitton of acronymns for the uninitiated.

Re: The Looming Battle Over AI Chips

#20

Post by joe_the_user »

Symmetry wrote: post #11
etaioinshrdlu wrote: post #8

It would be interesting to try emulate a many-core CPU as a GPU program and then run an OS on it. This sounds like a dumb idea, and it probably is. But consider a few things: * NVIDIA GPUs have exceptional memory bandwidth, and memory can be a slow resource on CPU based systems (perhaps limited by latency more than bandwidth) * The clock speed isn't that slow, it's in the GHz. Still one's clocks per emulated instruct…

Generally speaking emulating special purpose hardware in software slows things down a lot so I don't think that relying on a software branch predictor is going to result in performance anywhere close to what you'd see in, say, an ARM A53. And since you have to trade off clock cycles used in your branch predictor with clock cycles in your main thread I think it would be a net loss. Remember that even though NVidia cal…

Remember that even though NVidia calls each execution port a "Core" it can only execute one instruction across all of them at a time.

There are clever ways around this limitation, see links in my post this thread.

https://news.ycombinator.com/item?id=16892107

Post reply on HN