They acquired Bright Cluster Manager a few years ago, who would be next on their list to acquire? It seems like they want to provide customers with the whole stack.
Anthropic or Mistral and build AGI/ASI.
Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
251–260 of 340 posts
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#252Earlier quoted context omitted.
How can 4 bits possibly be enough? Are intermediate calculations done at a higher width and then converted down back to FP4?
- Training isn’t done at 4-bits, to date this small size has only been for inference. - Research for a while now has been finding that smaller weights are surprisingly effective. It’s kind of a counterintuitive result, but one way to think about it is there are billions of weights working together. So taken as a whole you still have a large amount of information.
> - Training isn’t done at 4-bits, to date this small size has only been for inference.
Wasn't there a paper from Microsoft two weeks ago or so where they trained on log₂(3) bits?
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#253Earlier quoted context omitted.
How can 4 bits possibly be enough? Are intermediate calculations done at a higher width and then converted down back to FP4?
There are research papers where even 1 bit (not floating point) was enough, with some quality loss. 4 bits is effectively 16 different float point numbers - 8 positive, 8 negative, no zero and no NaN/inf. 1 bit for sign and 3 bits for exponent, 0 bits for mantissa, mantissa is implied to be 4. It’s logarithmic - representing numbers in the range from -4^3 to 4^3, smallest numbers are 4^-3.
Thanks. First source i see for what fp4 is. Gotta say I'm surprised: I would have chosen to lose one value, but have a zero. (though I have no doubt those people are much more clever and knowledgeable than I am)
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#254Earlier quoted context omitted.
My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. NVIDIA could voluntarily open the standard to avoid this ligitation if they wanted to, though, and IMO it would be the smart thing to do, but almost every corporation in history has chosen the ligitation instead.
The cuda api is essentially open... Hip is basically a copy. CUDA is such a misnomer. Amd doesn't have tensorRT, cuDNN, cutlass, etc. Forcing Nvidia to make these work on AMD is like forcing Microsoft to make windows work on apple hardware... Not going to happen.
CUDA is not open. See what happened with ZLUDA.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#255Earlier quoted context omitted.
> My prediction is eventually there will be anti-trust ligitation, they will be required to open the CUDA standard, after which AMD will become a competitor. If AMD isn't a competitor before government intervention, I don't the government forcing nvidia to open up CUDA changes much. CUDA's moat isn't due to some secret sauce - nvidia put in the developer hours; and if AMDs CUDA implementation is still broken, people…
I don't understand AMD in this. Isn't it insanity that they're not throwing all they've got at their software stack?
It's a political problem. Good software engineers are paid more than good hardware engineers, but AMD management is unwilling to pay up to bring on good software engineers because then they'd also need to pay their hardware engineers more, otherwise the hardware engineers would be unsatisfied. If you check NVidia salaries online you'll see NVidia pays significantly more than AMD for both hardware and software engineers; it's a classic case of AMD management being penny-wise, pound-foolish.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#256Earlier quoted context omitted.
For better or worse, human psychology doesn't change that much - those historical standards very much apply today. For us, older folks, we've seen this 'new normal' several times already - it will end up as usual. There are no free lunches and as it appears to me that have not entered any permanently high plateau. It's even quite funny that ~100 years ago, we've had the previous big pandemic, and the biggest stock ma…
No, that’s objectively wrong. You simply haven’t seen retail involvement in the market on this level before, not even close. It’s a new precedent, the market has changed. To clarify, I’m not saying NVDA won’t crash from here or bear markets no longer exist. I’m simply saying that historic PE valuations are a poor metric for assessing the potential of a stock in todays market conditions.
Well, if you use the word "objectively", I would expect some numbers to back it up.
Yes, it's probably the first time that retail is allowed to trade options. But it's not the first time that retail is all in in stocks. I've tried to find a funny number to back it up - just check the Wiki on 1929 crash - https://en.wikipedia.org/wiki/Wall_Street_Crash_of_1929 - there was more money lent to 'small investors' so they can buy on margin ... than the entire amount of currency circulating at the time.
On average, all those retail guys will loose money - that's the sad truth. In the long term, the stocks simply follow the earnings - all other movements around this trends are pretty much a zero sum game - and most skilled operators are not loosing money in that game.
Price to earning ratio is just the number of years the company 'pays for itself' if you buy it. PER at 40s for big chunks of main indexes mean that either there will be tremendous progress in the economy that will boost the earnings or people are hoping to resell to a bigger fool.
Note: I work in finance, and I very much see the retail involvement in stocks. Hedge Funds and banks, are making a ton of money out of them, that's for sure.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#257My take from being at the keynote and the content I've seen so far at the conference is that Nvidia's is moving up the stack (like all good hardware vendors are prone to do). Obviously they are going to keep doing bigger. But the takeaway for me is that they are building "docker for llms" - NIM. They are building a container system where you can download/buy(?) NIMs and easily deploy them on their hardware. Going to…
Won't do anything to most consumer facing AI, the UI & convenience is already a major selling point. A bigger threat is that the feature the business is built around makes it into mainline software... there is no demand for (paid) background removal anymore as every iPhone can do it nowadays. Generally if whatever AI product you have can easily just be a feature in whatever application businesses already use, then yo…
Oh no, turns out the thing people really want from AIs is generality because everything else can be done in stupid software!
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#258Earlier quoted context omitted.
The cuda api is essentially open... Hip is basically a copy. CUDA is such a misnomer. Amd doesn't have tensorRT, cuDNN, cutlass, etc. Forcing Nvidia to make these work on AMD is like forcing Microsoft to make windows work on apple hardware... Not going to happen.
CUDA is not open. See what happened with ZLUDA.
I'm not sure your implication. My understanding of the project is AMD didn't want to invest in it anymore.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#259Earlier quoted context omitted.
You know what happens to companies that panic and throw all their resources into knee-jerk software projects? I don't, but I'd predict it is ugly. Adding more people to a bad project generally makes it worse. The issue that AMD has is they had a long period where they clearly had no idea what they were doing. You could tell just from looking at websites, CUDA pretty much immediately gets to "here is a library for FFT…
Look at AMDs vs Intel. They have now surpassed Intel in terms of CPUs sold and market cap. That was unthinkable even six, seven years ago. It makes perfect sense that, organisationally, they were focused on that battle. If you remember the Athlon days, AMD beat Intel before, but briefly. It didn't last. This time it looks like they beat Intel and have had the focus to stay. Intel will come back and beat them some cyc…
AMDs graphics was an acquisition though (ATI), and I understand that the company culture of that division might still be quite different.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#260I haven't listened to Jensen speak before, but am I the only one who thought the presentation wasn't very polished? Not a knock on anything he has accomplished, just an observation that sorta surprised me
If I am not mistaken he has said in some interviews he is not a big fan of public speaking.