Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

Post reply on HN
Print view View original

301–310 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#301

Post by lotsofpulp »

blagie wrote: post #278
bboygravity wrote: post #247

Earlier quoted context omitted.

Genuine question (I don't know much about cloud stuff): how is providing a cloud service/platform (at scale) even remotely as hard as designing, manufacturing and selling GPU's (including drivers and firmware) at massive scale? It feels like reading that setting up something like Facebook would be extremely challenging for a company like SpaceX.

Genuine answer: Setting up Facebook WOULD be extremely challenging for a company like SpaceX. There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software. To give an example in a closer domain: Look at how long Google lost money on cloud services through 2022 (over $15B in loses),…

>There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software.

One is a publicly listed business with as much of an objective look at real time "worth" as possible in today's world, and the other is a private business with confidential financials.

Seems like you would be unable to even calculate SpaceX's net worth, much less compare them to a business with the most objective measure of "worth".

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#302

Post by matt-p »

nemothekid wrote: post #194
dheera wrote: post #181

Earlier quoted context omitted.

My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. NVIDIA could voluntarily open the standard to avoid this ligitation if they wanted to, though, and IMO it would be the smart thing to do, but almost every corporation in history has chosen the ligitation instead.

> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…

It changes alot. It is not legal to make a 'CUDA' driver for an AMD GPU as Nvidia own cuda. You can see there was a open implementation of this that AMD sponsored until they got threatened with a lawsuit by Nvidia

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#303

Post by creshal »

imtringued wrote: post #238
creshal wrote: post #224

Earlier quoted context omitted.

Half the reason why they move back up to 8/16 bit is that current hardware doesn't properly support 4 bit floats, and you get better performance from the conversion. I think once this hardware hits, most of the computation will shift to native 4 bit just for efficency's sake. ...assuming the recent 1.58b paper doesn't render the entire float quantization approach obsolete by then.

The 1.58b approach is good for everyone including for quantization. It means that current quantization schemes have room for improvement.

It's good for everyone on the software side, but it's not so good for the hardware side, because it means that whatever you're designing now for tapeout in 6 months and release in 12 is going to be obsolete in 2 weeks.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#304

Post by Difwif »

bayindirh wrote: post #295
Difwif wrote: post #293

Earlier quoted context omitted.

Not sure it's fair to characterize modern LLMs as 'wasteful software development" or unoptimized. The implementations do quite an impressive level of optimization with what hardware is available. New theoretical methods w.r.t quantization represent most of our software optimization techniques and we're probably hitting the limit of that shortly with ternary or binary gates. To your point enthusiasts and developers wi…

Currently AI folks create bigger models and feed larger training data sets, because we still don't know the efficiency limits. IOW, currently we can't accelerate the learning beyond a certain point with less training data. This is esp. true on LLM/GenAI space. On the other areas where NNs are used, what I can see is training models with less data is not only plausible, but very possible, esp. in image processing, pro…

I just think we're talking about a completely different problem with AI optimization. There's $billions of effort and research that goes into AI optimization. Scaling model size and training set size happens because it's what the research and evidence tells us will improve model performance reliably. If we could reduce any of it for the same performance we would. Top model performance is an arms race and it's happening at every expense. The largest players are all shooting to beat GPT-4 or Claude Opus and achieve AGI (whatever that means).

This is very different than a program that requires zero research breakthroughs to dramatically improve and is simply slow and bloated because people have different priorities.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#305

Post by gammalost »

bayindirh wrote: post #263
gammalost wrote: post #261

I wonder when we as an industry will start to address the scaling issues in LLMs.It is obviously in Nvidias interest to keep pushing out bigger and better GPUs, but what is the collective interest? It is already proven that good language models are possible given enough resources. The challenge now is to put these models in a solution which do not require unfathomable amounts of resources for the average use cases.

Wasteful software development is easy and keeps momentum for development. As long as growth is king, quick and dirty will always beat well optimized and smaller systems. This is not a problem with AI only, but with every software we use. Only two groups try to optimize things and try to fit into smaller systems. Passionate programmers and people who is paid to do this (e.g.: phone manufacturers' software teams, etc.)…

I don't think it is thanks to wasteful software development. The libraries used for LLMs do a lot to squeeze out the full potential of GPUs.

I think it is more of an information problem. How can we store enough information in weights so that it is possible to train models without a budget similar to OpenAI

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#306

Post by svnt »

dkrich wrote: post #169
Blammar wrote: post #115

Earlier quoted context omitted.

NVDA's forward PE is ~37, about what it has been for the past ~5 years I've been tracking that. So it's not overpriced based on that metric. If you're convinced the stock is that overvalued, go short some or, if you like to live dangerously, buy some long-term put options (don't be an idiot and buy short-term options.) I have no idea if NVDA is like Cisco Systems in 2000, or if it's something unique. What I am aware…

We are at a very unique time. The stock market has basically been in a bull market for 15 years with some very short-lived sell-offs along the way. During that time we've had some incredible innovations such as the iPhone, FANG stock dominance and unprecedented profitability for years. You've also had three or four bona fide bubbles in that span, starting around 2017. First was Bitcoin along with the stock market as…

I’ll buy the credulity problem, and agree there is considerable risk in NVDA’s market position.

However they went up 8x because (neglecting crypto) they overnight transitioned from providing accessories to PC gamers and high end engineering workstations (both increasingly niche markets with tapering growth or decline) to being for the moment the only substrate of an entirely new consumer product segment that has seen the most rapid adoption of any new technology in the history of the world.

This could be the way things work now: the time constants shrink as the pipeline efficiency increases.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#307

Post by anon291 »

sgift wrote: post #269
coryrc wrote: post #258

Earlier quoted context omitted.

I'm not sure your implication. My understanding of the project is AMD didn't want to invest in it anymore.

IMHO there's reason to believe that was what was discussed here plays a role in that decision: https://news.ycombinator.com/item?id=39592689 - namely NVidia trying to forbid such APIs.

That has nothing to do with the API. The restriction there is you cannot use nvcc to generate nvidia bytecode, take that bytecode, decompile it, and translate it to another platform. This means that, if you use cuDNN, you cannot intercept the already-compiled neural network kernels and then translate those to AMD.

You can absolutely use the names of the functions and the programming model. Like I said, HIP is literally a copy. Llama.cpp changes to HIP with a #define, because llama.cpp has its own set of custom kernels.

And this is what I've said before, CUDA is hardly a moat. The API is well-known and already implemented by AMD. It's all the surrounding work: the thousands of custom (really fast!) kernels. The ease-of-use of the SDKs. The 'pre-built libraries for every use case'. You can claim that CUDA should be made open-source for competition, but all those libraries and supporting SDKs represent real work done by real engineers, not just designing a platform, but making the platform work. I don't see why NVIDIA should be compelled to give those away anymore than Microsoft should be compelled to support device driver development on linux.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#308

Post by blagie »

lotsofpulp wrote: post #301
blagie wrote: post #278

Earlier quoted context omitted.

Genuine answer: Setting up Facebook WOULD be extremely challenging for a company like SpaceX. There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software. To give an example in a closer domain: Look at how long Google lost money on cloud services through 2022 (over $15B in loses),…

>There's a reason Facebook is worth about 10x what SpaceX is worth, and most of that value doesn't come from the ability to build software. Facebook isn't even particularly good at building software. One is a publicly listed business with as much of an objective look at real time "worth" as possible in today's world, and the other is a private business with confidential financials. Seems like you would be unable to e…

SpaceX raised $750M at a valuation of $137B in January 2023.

A private investment at this scale should have a lot more transparency and due diligence than disclosures from a SEC disclosures. If I were investing $750M, I'd have engineers under NDA review SpaceX technologies, financial auditors, legal auditors, etc.

Secondary sales place it a little bit higher (but those typically have all the issues you describe).

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#309

Post by bayindirh »

Difwif wrote: post #304
bayindirh wrote: post #295

Earlier quoted context omitted.

Currently AI folks create bigger models and feed larger training data sets, because we still don't know the efficiency limits. IOW, currently we can't accelerate the learning beyond a certain point with less training data. This is esp. true on LLM/GenAI space. On the other areas where NNs are used, what I can see is training models with less data is not only plausible, but very possible, esp. in image processing, pro…

I just think we're talking about a completely different problem with AI optimization. There's $billions of effort and research that goes into AI optimization. Scaling model size and training set size happens because it's what the research and evidence tells us will improve model performance reliably. If we could reduce any of it for the same performance we would. Top model performance is an arms race and it's happeni…

> If we could reduce any of it for the same performance we would.

Nope, because all of them are harder than just going bigger.

> Top model performance is an arms race and it's happening at every expense.

This is also what I said. "Growth (in model performance) is king, so quick and dirty (going bigger) beating harder optimization efforts".

> This is very different than a program that... [Snipped for brevity]

Again this is what I said by "it keeps development momentum". Yes people have different priorities. Mostly money and fame in this point.

So, we don't disagree a bit.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#310

Post by jerf »

smallmancontrov wrote: post #54
jerf wrote: post #41

Earlier quoted context omitted.

This seems as good a place as any to be Corrected by the Internet, so... correct me if I'm wrong. Making a graphics chip that is as good as Nvidia: Very difficult. Huge moat, huge effort, lots of barriers, lots of APIs, lot of experience, lots of decades of experience to overcome. Making something that can run a NN: Much, much easier. I'd guess, start-up level feasible. The math is much simpler. There's a lot of it,…

I agree with you, but let me devil's advocate. After 10 years of pretending to care about compute, AMD has filled the industry with burned-once experts who, when weighing nvidia against competitors, instinctively include "likely boondoggle" against every competitor's quote because they've seen it happen, possibly several times. Combine this with nvidia's deep experience and and huge rich-get-richer R&D budget keeping…

It's true that nobody has beaten nVidia yet, and that is a valid data point I don't deny.

But (as a reply to some other repliers as well), AMD was also chasing them on the entire graphics stack as well as compute. That is trying to cross the moat. Even reimplementing CUDA as a whole is trying to cross a moat, even a smaller one.

But just implementing a chip that does AI, as it stands today, full stop, seems like it would be a lot easier. There's a lot of people doing it and I can't imagine they're all going to fail. I would consider by far the more likely scenario to be that the AI research community finds something other than neural nets to run on and thus the latest hotness becomes something other than a neural net and the chips become much less relevant or irrelevant.

And with the valuation of nVidia basically being based not on their graphics, or CUDA, but specifically just on this one feeding frenzy of LLM-based AI, it seems to me there's a lot of people with the motivation to produce a chip that can do this.

Post reply on HN