Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Looming Battle Over AI Chips

barrons.com

Post reply on HN
Print view View original

71–80 of 89 posts

Re: The Looming Battle Over AI Chips

#71

Post by droidist2 »

arca_vorago wrote: post #40
oneshot908 wrote: post #15

Earlier quoted context omitted.

It probably won't. For every oppressive move NVDA has made so far, there has been a swarm of low-information technophobe MBA sorts who eat their computational agitprop right up, some of them even fashion themselves as data scientists. More likely, NVDA continues becoming the Oracle of AI that everyone needs and everyone hates.

So is OpenCL dead? Because that's how everyone is talking. The tools you choose, and their licensing, matters!

It seems like people are placing their hopes into HIP more than OpenCL now when it comes to using chips from AMD and others.

Re: The Looming Battle Over AI Chips

#72

Post by taeric »

sanxiyn wrote: post #70
taeric wrote: post #52

Earlier quoted context omitted.

This somewhat blows my mind. Yes, it is impressive. However, the work that Nuance and similar companies used to do are still competitive, just not getting near the money and exposure. I remember over a decade ago, they even had mood analysis they could apply to listening to people. Far from new. Is it truly more effective or efficient nowadays? Or just getting marketed by companies you've heard of?

It is truly better. Objective metrics (such as word error rate) don't lie. You can argue whether it makes sense to use, say, 100x compute to get 2x less error, but that's a different argument; I don't think anyone is really disputing improved quality.

Do you have a good comparison point? And not, hopefully, comparing to what they could do a decade ago. I'm assuming they didn't sit still. Did they?

I question whether it is just 100x compute. Feels like more, since naturally speaking and friends didn't hog the machine. Again, over a full decade ago.

More, the resources that Google has to throw at training are ridiculous. Well over 100x what was used to build the old models.

None of this is to say we should pack up and go back to a decade ago. I just worry that we do the opposite; where we ignore progress that was made a decade ago in favor of the new tricks alone.

Re: The Looming Battle Over AI Chips

#73

Post by MR4D »

Back in the day, there was a 386. And also a 387 coprocessor to have the tougher math bits.

Then came a 486 and it got integrated again.

But during that time, the GPU split off. Companies like ATI and S3 began to dominate, and anyone wanting a computer with decent graphics had one of these chips in their computer.

Fast forward several years, and Intel again would bring specialized circuitry back into their main chips, although this time for video.

Now we are just seeing the same thing again, but this time it’s an offshoot of the GPU instead of the CPU. Seems like the early 1990’s again, but the acronyms are different.

Should be fun to watch.

Re: The Looming Battle Over AI Chips

#74

Post by bogomipz »

maffydub wrote: post #37
bogomipz wrote: post #17

The article states: >"LeCun and other scholars of machine learning know that if you were starting with a blank sheet of paper, an Nvidia GPU would not be the ideal chip to build. Because of the way machine-learning algorithms work, they are bumping up against limitations in the way a GPU is designed. GPUs can actually degrade the machine learning’s neural network, LeCun observed. “The solution is a different architec…

I went to a talk from the CTO of Graphcore ( https://www.graphcore.ai/ ) on Monday. They are designing chips targeted at machine learning. As I understood it, their architecture comprises - lots of "tiles" - small processing cores with collocated memory (essentially DSPs) - very high bandwidth (90TB/s!) switching fabric to move data between tiles - "Bulk Synchronous Parallel" operation, meaning that the tiles do thei…

Thanks for the detailed response and link! Cheers.

Re: The Looming Battle Over AI Chips

#75

Post by jacksmith21006 »

taeric wrote: post #52
jacksmith21006 wrote: post #29

The new Google Speech solution is the perfect example on why Google had to do their own silicon. Doing speech with 16k samples a second through a NN and keep at a reasonable cost is really, really difficult. The old way was far more power efficient and if you are going to use this new technique which gets you a far better result and do it at a reasonable cost you have to go all the way down into the silicon. Here lis…

This somewhat blows my mind. Yes, it is impressive. However, the work that Nuance and similar companies used to do are still competitive, just not getting near the money and exposure. I remember over a decade ago, they even had mood analysis they could apply to listening to people. Far from new. Is it truly more effective or efficient nowadays? Or just getting marketed by companies you've heard of?

"Nuance and similar companies used to do are still competitive"

Surprised. Curious if you can compare the inference per joules of Google 1st gen TPUs compared? Google shared a paper and the numbers are pretty impressive and was not aware of anyone else close to the gen 1 TPUs?

Here is the paper that you can use for the TPU side. Love so see someone else in the ball park? We really do not want just one company but competition.

https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf

Re: The Looming Battle Over AI Chips

#76

Post by jacksmith21006 »

taeric wrote: post #72
sanxiyn wrote: post #70

Earlier quoted context omitted.

It is truly better. Objective metrics (such as word error rate) don't lie. You can argue whether it makes sense to use, say, 100x compute to get 2x less error, but that's a different argument; I don't think anyone is really disputing improved quality.

Do you have a good comparison point? And not, hopefully, comparing to what they could do a decade ago. I'm assuming they didn't sit still. Did they? I question whether it is just 100x compute. Feels like more, since naturally speaking and friends didn't hog the machine. Again, over a full decade ago. More, the resources that Google has to throw at training are ridiculous. Well over 100x what was used to build the old…

The thing is it is not simply the training but the inference aspect would have require an incredible amount of compute compared to the old way of doing it.

Hope Google will do a paper like they did with the Gen 1 TPUs. Would love to see the difference in terms of joules per word spoke.

Re: The Looming Battle Over AI Chips

#77

Post by jacksmith21006 »

emcq wrote: post #32

There is certainly a lot of hype around AI chips, but I'm very skeptical of the reward. There are several technical concerns I have with any "AI" chip that ultimately leave you with something more general purpose (and not really an "AI" chip, but good at low precision matmul): * For inference, how do you efficiently move your data to the chip? In general most of the time is spent in matmul, and there are lots of exci…

Here is a great paper that might help.

https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf

But also this is a great example of why these new "AI" chip matters. This would NOT be possible with the Google chips.

https://cloudplatform.googleblog.com/2018/03/introducing-Clo...

I do wish Google shares more detail. Specifically how they are doing 16k samples a second through a NN. That has a lot of application beyond speech.

Some things Google will not share and this might be one of them but we can hope.

Re: The Looming Battle Over AI Chips

#78

Post by emcq »

joshuamorton wrote: post #64
emcq wrote: post #63

Earlier quoted context omitted.

At a high level there is a design tradeoff where you put your transistors for a given chip. For a dense linear algebra/tensor processor, it basically comes down to using your transistors for memory or compute. GPUs (and DSPs) historically are way on the compute side. You get kilobytes of on chip memory, and really fat parallel buses to off chip RAM. On the other end, you have some chips that put more memory near the…

>At Google/FB scale tweaks to improve cost may make sense but it doesnt seem like architecturally speaking there is some major design decision being left on the table anymore (I'd love to hear a specific counterpoint though!). I'd expect if I knew any, they'd be under NDA. I'll just point out that a GPU, even one with specific "ML cores" as NVIDIA calls them, is going to have a bunch of silicon that is being used ine…

There are many researchers not bound by NDAs making novel chip architectures, some of which are on HN.

Secondly, GPUs don't really have very much hardware specialized for graphics anymore. If we called it a TPU I'm not sure you'd be making the same point :P At a company like MS/FB/Google where you don't need to leverage selling the same chip for gaming/vr/ML/mining for economy of scale, like you said you can reduce your transistor count and have the same compute fabric. This would reduce your idle power consumption through leakage but you wouldn't expect a huge drop in power during active compute. Because the smaller precision compute ends up increasing the compute per byte, you either need to find more parallelism to RAM, get faster RAM, reduce the clockrate of compute, or reduce the number of compute elements to find a balanced architecture with lower precision. If you just shrink the number of compute elements - voila! - you're close to what NVIDIA is doing with ML cores.

Re: The Looming Battle Over AI Chips

#79

Post by jacksmith21006 »

m3kw9 wrote: post #30
jacksmith21006 wrote: post #28

Earlier quoted context omitted.

Do not see why Google would care what Nvidia is doing? Why would they?

If they do it better and cheaper they care

Google is doing the entire stack so kind of hard for Nvidia to have a competitive product to the TPUs.

But also Google is already doing the 2nd generation of the TPUs and would expect a third.

The data of chips coming from 3rd parties are probably numbered.

The dynamics of the industry changed where now the companies that are buying the chips have skin in the game with the cost of running the chips. Versus the old days Intel sold a chip to Dell who then sold it to someone else.

It just makes sense for Google to do their own silicon at this point as saves them a ton of money.

But also Google has the data to improve the chip that Nvidia just does not have.

Re: The Looming Battle Over AI Chips

#80

Post by jacksmith21006 »

The dynamics of the chip industry have completely changed. Use to be a chip company like Intel sold their chips to a company like Dell that then sold the server with the chip to a business which ran the chip and paid the electric bill.

So the company that made the chip had no skin in the game with running the chip or the cost of the electricity to run it.

Today we have massive cloud with Google and Amazon and lowering the cost of running their operations goes a long way unlike the days of the past.

This is why we will see more and more companies like Google create their own silicon which has already started and well on it's way.

Not only the TPUs but Google has created their own network processors as they quietly hired away the Lanai team years ago.

https://www.informationweek.com/data-centers/google-runs-cus...?

Also this article helps explain why Google built the TPUs.

https://www.wired.com/2017/04/building-ai-chip-saved-google-...

Post reply on HN