Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

271–280 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#271
post #215

Earlier quoted context omitted.

I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?

You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT…

> AMD would explain that ROCM is an abbreviation of the ROCm Software platform or something unspeakably stupid. And that your graphics card wasn't supported.

If even that. A few years ago they managed to break basic machine learning code on the few commonly-used consumer GPUs that were officially supported at the time, and it was only after several months of more or less radio silence on the bug report and several releases that they declared those GPUs were no longer officially supported and they'd be closing the bug report: https://github.com/ROCm/ROCm/issues/1265

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#272
post #76

Earlier quoted context omitted.

And MS and everyone else have plenty of interest in helping AMD commodify CUDA compatibility.

It's so weird it's taking them so long, because as far as anyone can tell AMD is mostly competent enough to make GPUs within some percentage points of Nvidia, the "breadth of complexity" in what these things do at the end of the day is ... rather underwhelming, the software stack may appear to be changing all the time but is also distinctly JavaScript-frotend-esque... is there an insider that knows what the holdup is…

AMD pays very little to its SWEngs (principal engineer in SFBA for ~200k), so they can't attract top end people in SW to implement what they need. Semi companies are used to pay HW engineers peanuts and that doesn't work in SW.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#273

Double digit peta flop mass produced. "The computing power needed to replicate the human brain’s relevant activities has been estimated by various authors, with answers ranging from 10^12 to 10^28 FLOPS." Petaflop is 10^15 Crazy times.

I’ll be happy with this if we use it to design viable fusion power plants. And I’ll be severely disappointed if it’s mostly used for ad targeting.

Fusion plants will be used to power ad targeting Nanoprobes.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#274
post #270

Earlier quoted context omitted.

Won't do anything to most consumer facing AI, the UI & convenience is already a major selling point. A bigger threat is that the feature the business is built around makes it into mainline software... there is no demand for (paid) background removal anymore as every iPhone can do it nowadays. Generally if whatever AI product you have can easily just be a feature in whatever application businesses already use, then yo…

> here is no demand for (paid) background removal anymore as every iPhone can do it nowadays. Proper background removal of even remotely complex content is still in demand, especially on a large scale. I doubt you'd use an iPhone to work on >100 of images per second.

Matthew Bryant would disagree.

https://findthatmeme.com/blog/2023/01/08/image-stacks-and-ip...

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#275

Earlier quoted context omitted.

Doesn’t matter how good your in-house tech team was, your company still outsourced it to cloud infra. That’s what Nvidia faces. Doesn’t matter how good the current in-house teams are using direct hardware, the trend in corporate is shift to a vendor (Google/AWS). Nvidia can watch this inevitable shift or get ready to offer itself as a platform too.

I get it, the service model always shines the brightest in the eye of the revenue calculator. But I have immediate skepticism they’ll be able to execute at a competitive level. Their core competency has always been manufacture and production, not service-based things. It’s a big rock to push up a tall hill.

They have been building partnerships with ISP's and service providers all over the world with their GeForce now game streaming service. They could continue and expand this by providing a similar backend for LLM services.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#276

Earlier quoted context omitted.

Tell me more about why you believe their stock is hilariously overvalued.

They are priced as if they are the only ones who are capable of creating chips that can crunch LLM algos. But AMD, Google, Intel, and even Apple are also capable. Apple is in talks with Google to bring Gemini to the iPhone, and it will obviously also be on android phones. So almost every phone on earth is poised to be using Gemini in the near future, and Gemini runs entirely on Google's own custom hardware (which is…

AMD is even more hilariously overvalued, currently at 360 PE

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#277

Earlier quoted context omitted.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

I share your intuition, perhaps unfairly, that it's indeed not as hard in absolute terms. However, it certainly requires a different set of skills and organisational practices. Just because an organisation is extremely good at one thing doesn't mean it can easily apply that to another field. I would guess that SpaceX probably does have the talent on hand to throw together a Facebook clone, but equally I think they wo…

Well, motivation would be an obvious thing lacking. People who want to work on rockets, I would guess, would find working on facebook to be a “boring” solved problem.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#278

Earlier quoted context omitted.

I get it, the service model always shines the brightest in the eye of the revenue calculator. But I have immediate skepticism they’ll be able to execute at a competitive level. Their core competency has always been manufacture and production, not service-based things. It’s a big rock to push up a tall hill.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

Genuine answer: Setting up Facebook WOULD be extremely challenging for a company like SpaceX. There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software.

To give an example in a closer domain: Look at how long Google lost money on cloud services through 2022 (over $15B in loses), and now only makes money by creative accounting (bundling "cloud services" together versus breaking out GCP; Microsoft does something similar with Office 365 and Azure).

Like many potential customers, I would not consider GCP because:

1) Google "support" is a buggy, automated algorithm which randomly thwacks customers on the head

2) Google randomly discontinues products

3) I've seen a half-dozen to a dozen instances where buying from Google was penny-wise and pound-foolish, and so have many other engineers I've worked with.

Google's overall attitude is that I'm a statistic defined by my value to Google. Google can and will externalize costs onto me. That attitude is 100% right for adwords and search, which are defined by margins, but not for something like GCP. If I am going with a cloud service/platform, I'll go with Amazon, Microsoft, or just about anyone else, for that matter.

That's not that Google is a bad company. Google actually did have the skill set to build the software and data centers for a very, very good cloud provider. It's just that Google's core competencies lie very far from providing reliable service to customers, customer support, or all the things which go into providing me with stability and business continuity.

"Fixing" this would require a wholesale culture, value, and attitude change, and developing a core competency very far from what Google is good at.

I put "fixing" in quotes since if you develop too many core competencies, you usually stop being good at any of them. Focus is important, and there's a reason many businesses spin out units outside of their domains of focus. If Google is able to become good at this, but in the process loses their edge in their current core competencies, that's probably a bad deal.

FWIW: I haven't yet formed an opinion on NVidia's cloud strategy. However, their core competencies appear to be very much in the "hard" domains like silicon, digital design, machine learning rather than "soft" ones. Another relevant example for what can happen when hard skills are de-emphasized at engineering-driven companies is Boeing (if you've been following recent stories; if not, watch a documentary).

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#279
post #183
post #178

> “Nvidia … is becoming less of a mercenary chip provider and more of a platform provider, like Microsoft or Apple, on which other companies can build software. I can understand from a growth perspective why it’s more profitable for Nvidia if it can become more of a platform service for AI. However, that’s difficult to balance that and partnerships the company already has with AWS and Microsoft. I’d expect to see som…

I think they're planning for a world where half their customers (hyperscalers) just use GPUs and CUDA while the other half (the long tail) use more profitable, higher-level parts of the platform. They don't have the leverage to force customers one way or the other. It would be easier to just sell GPUs, but they know that sophisticated customers can switch to other chips while the platform provides lock-in for smaller…

The hyperscalers aren't going to be on CUDA for long, Nvidia are taking too much of a cut. Google run tensor chips, Amazon and Microsoft both have their own accelerators in the works. They are massively incentivised to do so right now.

Now they do all re-sell Nvidia GPUs in their cloud businesses, but there the cost will be passed very directly on to infrastructure customers, who will see the competing higher level services from those cloud providers (hosted models) likely at lower or competitive prices, and it's going to be harder to justify renting CUDA cores for custom software.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#280
post #215

Earlier quoted context omitted.

You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT…

Pytorch has been supporting rocm for all last 2 years

I'd add quotes there:

Pytorch has been "supporting" rocm for all last 2 years

Post reply on HN