Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Looming Battle Over AI Chips

barrons.com

Post reply on HN
Print view View original

51–60 of 89 posts

Re: The Looming Battle Over AI Chips

#51

Post by joshuamorton »

oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

I'm a little confused here, are you saying that ML ASICs can't beat compute per $ of GPUs? That seems, on its face, to be a ridiculous assertion, so I'm confused where I'm misunderstanding you.

Re: The Looming Battle Over AI Chips

#52

Post by taeric »

jacksmith21006 wrote: post #29

The new Google Speech solution is the perfect example on why Google had to do their own silicon. Doing speech with 16k samples a second through a NN and keep at a reasonable cost is really, really difficult. The old way was far more power efficient and if you are going to use this new technique which gets you a far better result and do it at a reasonable cost you have to go all the way down into the silicon. Here lis…

This somewhat blows my mind. Yes, it is impressive. However, the work that Nuance and similar companies used to do are still competitive, just not getting near the money and exposure.

I remember over a decade ago, they even had mood analysis they could apply to listening to people. Far from new. Is it truly more effective or efficient nowadays? Or just getting marketed by companies you've heard of?

Re: The Looming Battle Over AI Chips

#53

Post by guipsp »

mattnewton wrote: post #46
oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

s /Jensen / nvidia ?

Jensen refers to NVDA's President & CEO, Jensen Huang

Re: The Looming Battle Over AI Chips

#54

Post by oneshot908 »

zantana wrote: post #36
deepnotderp wrote: post #18

Do people think that nobody at nVidia has ever heard of specialized deep learning processors? 1. Volta GPUs already have little matmul cores, basically a bunch of little TPUs. 2. The graphics dedicated silicon is an extremely tiny portion of the die, a trivial component (source: Bill Dally, nVidia chief scientist). 3. Memory access power and performance is the bottleneck (even in the TPU paper), and will only continu…

Is there an easy way that Nvidia can cripple their graphics targeted cards so they can't be used for GPCPU? I'm thinking back to strategies like the 486SX https://en.wikipedia.org/wiki/Intel_80486SX

Yes, they've been trying to do that in various forms since the 2009 Fermi GPU GTX 480 which crippled FP64. It never works. But it does create technical debt to work around their nonsense.

So unless they cripple CUDA altogether, there will always be efficient workarounds (arguably DirectX or OpenGL programmable shaders in the very worst-case scenario). They even gave up on doing so for GTX Titan Black and then resumed with Maxwell. Currently, I would not be surprised that the lack of a true consumer Volta GPU is their only play at crippling consumer Volta by making it effectively nonexistent or $3000 for the GTX Titan V.

What they could do across the board is hamfistedly disable the deep learning frameworks on GeForce. That would probably stop 90% of amateur hour data science on GeForce. But the remaining 10% would just recompile them without the cripple code in violation of some sort of scary EULA clause against doing so and requiring such cripple code in all HPC/AI applications. I would love to see them try this - they'll pry my FP32 MADs (which is the core operation of AI/ML as well as vertex and pixel shaders) from my cold dead consumer GPU desktop.

I don't think they'll do that though. They know the low-end is the entry point to their ecosystem. They just want to force people to graduate into the high-end after hooking them. Not that you have to: multiplication and addition want to be free.

Re: The Looming Battle Over AI Chips

#55

Post by hueving »

lostgame wrote: post #19
oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

FYI I think your comment is informative and I understood a lot of it but that's a shitton of acronymns for the uninitiated.

FB: Facebook

GOOG: Google

AAPL: Apple

AMZN: Amazon

BIDU: Baidu

ASIC: Application specific integrated circuit

GPU: Graphics processing unit

CUDA: Compute-unified device architecture

EULA: End-user license agreement

weed: marijuana

HPC: High-performance computing

NVDA: Nvidia

HW: hardware

Re: The Looming Battle Over AI Chips

#56

Post by oneshot908 »

joshuamorton wrote: post #51
oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

I'm a little confused here, are you saying that ML ASICs can't beat compute per $ of GPUs? That seems, on its face, to be a ridiculous assertion, so I'm confused where I'm misunderstanding you.

No, they can't when you can get tens of TFLOPS per GPU for Now when a Tesla GPU costs $9-10K instead of In contrast, if you can right-size the transistor count for expensive and very specific workloads across 100K+ machines like companies with big datacenters and big data sets can do, I see an opportunity for all of the BigCos to build custom ASICs for their siloed data sets and graphs. That's what GOOG is doing and it seems to be working for them so far. FB is now hiring to do the same thing I suspect.

Re: The Looming Battle Over AI Chips

#57

Post by zantana »

oneshot908 wrote: post #14

If you're FB, GOOG, AAPL, AMZN, BIDU, etc, this strategy makes sense because much like they have siloed data, they also have siloed computation graphs for which they can lovingly design artisan transistors to make the perfect craft ASIC. There's big money in this. Or you can be like BIDU, buy 100K consumer GPUs, and put them in your datacenter. In response, Jensen altered the CUDA 9.1 licensing agreement and the EULA…

I just had an image of pranksters "swatting" datacenters by feeding false information to Nvidia that they are using consumer cards.

Re: The Looming Battle Over AI Chips

#58

Post by Symmetry »

joe_the_user wrote: post #20
Symmetry wrote: post #11

Earlier quoted context omitted.

Generally speaking emulating special purpose hardware in software slows things down a lot so I don't think that relying on a software branch predictor is going to result in performance anywhere close to what you'd see in, say, an ARM A53. And since you have to trade off clock cycles used in your branch predictor with clock cycles in your main thread I think it would be a net loss. Remember that even though NVidia cal…

Remember that even though NVidia calls each execution port a "Core" it can only execute one instruction across all of them at a time. There are clever ways around this limitation, see links in my post this thread. https://news.ycombinator.com/item?id=16892107

Those are some really clever ways to make sure that all the threads in your program are executing the same instruction, but it doesn't get around the problem. Thanks for linking that video, though.

Re: The Looming Battle Over AI Chips

#59

Post by Barjak »

If I were better credentialed, I would definitely be looking to get into semiconductors right now. It's an exciting time in terms of manufacturing processes, and I think some of the most interesting and meaningful optimization problems ever formulated come from semiconductor design and manufacturing, not to mention the growing popularity of specialized hardware.

I would tell a younger version of myself to focus your education on some aspect of the semiconductors industry.

Re: The Looming Battle Over AI Chips

#60

Post by twtw »

madengr wrote: post #4
willvarfar wrote: post #2

I just seem to bump into a paywall. The premise from the title seems plausible, although NVIDIA seems to be catching up again fast.

I was impressed enough with their CES demo to buy some stock. Isn’t the Volta at 15E9 transistors? It’s at the point only the big boys can play in that field due to fab costs, unless it’s disrupted due to some totally new architecture. First time on HN I can read a paywalled article, as I have a Barron’s print subscription.

21e9 transistors.
Post reply on HN