Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

Post reply on HN
Print view View original

281–290 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#281

Post by jerska »

flessner wrote: post #220
__mharrison__ wrote: post #204

My take from being at the keynote and the content I've seen so far at the conference is that Nvidia's is moving up the stack (like all good hardware vendors are prone to do). Obviously they are going to keep doing bigger. But the takeaway for me is that they are building "docker for llms" - NIM. They are building a container system where you can download/buy(?) NIMs and easily deploy them on their hardware. Going to…

Won't do anything to most consumer facing AI, the UI & convenience is already a major selling point. A bigger threat is that the feature the business is built around makes it into mainline software... there is no demand for (paid) background removal anymore as every iPhone can do it nowadays. Generally if whatever AI product you have can easily just be a feature in whatever application businesses already use, then yo…

> there is no demand for (paid) background removal anymore as every iPhone can do it nowadays.

PhotoRoom begs to disagree with this statement.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#282

Post by 55555 »

dkrich wrote: post #169
Blammar wrote: post #115

Earlier quoted context omitted.

NVDA's forward PE is ~37, about what it has been for the past ~5 years I've been tracking that. So it's not overpriced based on that metric. If you're convinced the stock is that overvalued, go short some or, if you like to live dangerously, buy some long-term put options (don't be an idiot and buy short-term options.) I have no idea if NVDA is like Cisco Systems in 2000, or if it's something unique. What I am aware…

We are at a very unique time. The stock market has basically been in a bull market for 15 years with some very short-lived sell-offs along the way. During that time we've had some incredible innovations such as the iPhone, FANG stock dominance and unprecedented profitability for years. You've also had three or four bona fide bubbles in that span, starting around 2017. First was Bitcoin along with the stock market as…

Good post, thanks.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#283

Post by e4325f »

AYBABTME wrote: post #199
nemothekid wrote: post #194

Earlier quoted context omitted.

> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…

I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?

They did buy Nod.ai recently

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#284

Post by fragmede »

throwaway2037 wrote: post #268
bboygravity wrote: post #247

Earlier quoted context omitted.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

I like this question. You raise a very good point. Occam's Razor tells me the simplest explanation is "core competency". Running AI-SaaS is just a very different business from creating GPUs (including the required software ecosystem). As a counterpoint: Look at Microsoft. Traditionally, they have been pretty good at writing (and making money from) enterprise software, but not very good at hardware. (XBox is the one B…

Except for the place where they're good at hardware, they're not good at hardware? I mean that's true, but a bit of a twist of logic, wouldn't you say?

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#285

Post by terramex »

tmalsburg2 wrote: post #252
WhitneyLand wrote: post #138

Earlier quoted context omitted.

- Training isn’t done at 4-bits, to date this small size has only been for inference. - Research for a while now has been finding that smaller weights are surprisingly effective. It’s kind of a counterintuitive result, but one way to think about it is there are billions of weights working together. So taken as a whole you still have a large amount of information.

> - Training isn’t done at 4-bits, to date this small size has only been for inference. Wasn't there a paper from Microsoft two weeks ago or so where they trained on log₂(3) bits? Edit: https://arxiv.org/pdf/2402.17764.pdf

They don't "train on log₂(3) bit". Gradients and activations are still calculated at full (8-bit) precision and weights are quantised after every update.

This makes network minimise loss not only with regard to expected outcome but also minimises loss resulting from quantisation. With big networks their "knowledge" is encoded in relationships between weights, not in their absolute values so lower precision work well as long as network is big enough.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#286

Post by fragmede »

blagie wrote: post #278
bboygravity wrote: post #247

Earlier quoted context omitted.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

Genuine answer: Setting up Facebook WOULD be extremely challenging for a company like SpaceX. There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software. To give an example in a closer domain: Look at how long Google lost money on cloud services through 2022 (over $15B in loses),…

Eh I mean Google moved GCP's revenue around because Microsoft was doing that to make Azure look bigger than GCP. If you can't beat em, join em. Google's got long term contracts with a lot of companies and the government, so GCP isn't going to shut down anytime soon. Their consumer products division has problems with product longevy, but we're not paying them corporation level money or signing serious contracts when buying a Stadia subscription. So it's just business.

What I've heard is Azure is a pain in the ass, and things take three times as long to set up there, for some reason. There's also Oracle cloud but you hear way less about them. out there.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#287

Post by michaelt »

bboygravity wrote: post #247
joshellington wrote: post #206

Earlier quoted context omitted.

I get it, the service model always shines the brightest in the eye of the revenue calculator. But I have immediate skepticism they’ll be able to execute at a competitive level. Their core competency has always been manufacture and production, not service-based things. It’s a big rock to push up a tall hill.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

It's certainly possible to start new cloud providers - there are a bunch of smaller-scale VM providers. In the GPU cloud business there's companies like lambdalabs and runpod.

But the fattest profit margins are in selling to big corporations. Big corporations who already have accounts with the likes of AWS. They already have billing set up, and AWS provides every cloud service under the sun. Container registry? Logging database? Secret management? Private networks? Complicated-ass role management? Single-sign-on integration? SOC2/PCI/HIPAA compliance? A costs explorer with a full API? Everything a growing bureaucracy could need. Getting your GPU VMs from your existing cloud provider is the path of least resistance.

The smaller providers often compete by having lower prices - but competing on cost isn't generally a route to fat profit margins. And will folks at big corporations care that you're 30% cheaper, when they're not spending their own money?

nvidia could definitely launch a focused cloud product, that competes on price - but would they be happy doing that? If they want to get into the business of offering everything from SAML logon integration to a managed labelling workforce with folks fluent in 8 languages - that could be a great deal of work.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#288

Post by selvan »

cebert wrote: post #178

> “Nvidia … is becoming less of a mercenary chip provider and more of a platform provider, like Microsoft or Apple, on which other companies can build software. I can understand from a growth perspective why it’s more profitable for Nvidia if it can become more of a platform service for AI. However, that’s difficult to balance that and partnerships the company already has with AWS and Microsoft. I’d expect to see som…

Microsoft and AWS would have a partnership with AMD/Intel for their GPUs, if those are capable and widely used as Nvidia's.

Microsoft has partnetship with OpenAI and also with Mistral.

Present convenience may not hold true in future. Nvidia knows that well.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#289

Post by actionfromafar »

fragmede wrote: post #274
dig1 wrote: post #270

Earlier quoted context omitted.

> here is no demand for (paid) background removal anymore as every iPhone can do it nowadays. Proper background removal of even remotely complex content is still in demand, especially on a large scale. I doubt you'd use an iPhone to work on >100 of images per second.

Matthew Bryant would disagree. https://findthatmeme.com/blog/2023/01/08/image-stacks-and-ip...

Holy Telephony Batman! That's both insane and perfectly rational at the same time.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#290

Post by nebula8804 »

imtringued wrote: post #234
elbear wrote: post #211

Earlier quoted context omitted.

Why's that?

Because they didn't go to uni when hardware-software-codesign was being taught.

What unis would that include? Isn't ATI Canadian? Therefore i'd expect lots of UToronto and Waterloo people there. Aren't they some of the best in this field?
Post reply on HN