Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

cnbc.com

Post reply on HN
Print view View original

231–240 of 340 posts

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#231

Post by rytill »

TheAlchemist wrote: post #165
rytill wrote: post #159

Earlier quoted context omitted.

Then be the options seller. You can sell cash secured puts, or a put credit spread, or a call credit spread. Calls are even more expensive than puts right now.

Selling options is an even worse idea. Frankly, I don't understand why we made it possible for individuals to gamble by selling options. As Charlie Munger used to say, Wall Street will sell shit as long as shit can be sold.

You would be right about selling naked options. But call/put credit spreads have bounded downside, just like buying options.

Selling cash secured puts or selling covered calls would be less risky than just holding stock.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#232

Post by jazzyjackson »

_factor wrote: post #162
AlexandrB wrote: post #82

Earlier quoted context omitted.

There's another case for pessimism as well: cost. It's possible that many AI applications aren't worth the money required for the extra compute. AI-enhanced search comes to mind here: how is Microsoft going to monetize users of Copilot in Bing to justify the extra cost? Right now a lot of this stuff is heavily subsidized by VCs or the MSFTs of the world, but when it comes time to make a profit we'll see what actually…

Better question: why does a simple search for “What color is a labrador retriever” require any compute time when the answer can be cached? This is a simple example, but 90% of my searches don’t require an llm to process a simple question.

One time I came across a git repo that let me download a gigabyte of prime numbers and I thought to myself, is that more or less efficient than me running a program locally to generate a gigabyte of prime numbers?

The compute for a direct answer like that is fractions of a penny, it might be better to create answers on the fly than store an index of every question anyone has asked (well, that's essentially what the weights are after all)

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#233

Post by jazzyjackson »

jedberg wrote: post #87
tamimio wrote: post #66

I think at this point, they should stop making it video “cards” but rather video “stations”, a full tower station with power supply and one giant “card” inside with proper cooling, etc., might also justify the crazy prices anyway.

https://lambdalabs.com/gpu-workstations/vector

I love that "we installed a working python environment for you" is a front-page value-add

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#234

Post by imtringued »

elbear wrote: post #211
tempaccount420 wrote: post #201

Earlier quoted context omitted.

Hardware people don't get along very well with software people.

Why's that?

Because they didn't go to uni when hardware-software-codesign was being taught.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#235

Post by blitzar »

nemothekid wrote: post #194
dheera wrote: post #181

Earlier quoted context omitted.

My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. NVIDIA could voluntarily open the standard to avoid this ligitation if they wanted to, though, and IMO it would be the smart thing to do, but almost every corporation in history has chosen the ligitation instead.

> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…

Getting this working might be worth a trillion $ to AMD - they should be doing more than just waiting for a bootstrapped startup to debug their drivers for them.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#237

Post by imtringued »

AYBABTME wrote: post #199
nemothekid wrote: post #194

Earlier quoted context omitted.

> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…

I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?

You have to remember that this only applies to cheap consumer GPUs, they tend to support their datacenter GPUs better. When you consider that Ryzen AI already eats the AI inference lunch, having better GPUs with better software only threatens to cannibalize their data center GPU offering. Given enough time nobody will care about using AMD GPUs for AI.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#238

Post by imtringued »

creshal wrote: post #224
jxy wrote: post #179

Earlier quoted context omitted.

curiously, what real world usage actually uses FP4? AFICT, most of the LLMs still use BF16, and even the quantizations down to 4bits and 2bits end up back to 16bit or INT8 for actual computations.

Half the reason why they move back up to 8/16 bit is that current hardware doesn't properly support 4 bit floats, and you get better performance from the conversion. I think once this hardware hits, most of the computation will shift to native 4 bit just for efficency's sake. ...assuming the recent 1.58b paper doesn't render the entire float quantization approach obsolete by then.

The 1.58b approach is good for everyone including for quantization. It means that current quantization schemes have room for improvement.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#239

Post by imtringued »

acchow wrote: post #148
WhitneyLand wrote: post #138

Earlier quoted context omitted.

- Training isn’t done at 4-bits, to date this small size has only been for inference. - Research for a while now has been finding that smaller weights are surprisingly effective. It’s kind of a counterintuitive result, but one way to think about it is there are billions of weights working together. So taken as a whole you still have a large amount of information.

Intuitively, there is a ton of redundancy and we still have a long way we can still compress things.

Each token is represented by a vector of 4096 floats. Of course there is redundancy.

Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’

#240

Post by elcomet »

roenxi wrote: post #215
AYBABTME wrote: post #199

Earlier quoted context omitted.

I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?

You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT…

Pytorch has been supporting rocm for all last 2 years
Post reply on HN