They acquired Bright Cluster Manager a few years ago, who would be next on their list to acquire? It seems like they want to provide customers with the whole stack.
Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
91–100 of 340 posts
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#92Earlier quoted context omitted.
He always does that. They stack up a bunch of special case features like sparsity that most people don't use in practice to get these unrealistic numbers. It'll be faster, certainly, but 30x will only be achievable in very special cases I'm sure.
Isn't sparsity almost always a win at this point? Making everything fully connected is a major waste.
The kind of sparsity that the hardware supports is not fully general. I'm not aware of any large models trained using it. Maybe they are all leaving 2x perf on the table for no reason, but maybe not. I don't think sparsity is really proven to be "almost always a win" for training.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#93I think at this point, they should stop making it video “cards” but rather video “stations”, a full tower station with power supply and one giant “card” inside with proper cooling, etc., might also justify the crazy prices anyway.
Isn't that just a computer? Or an eGPU, if it doesn't contain the rest of the computer?
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#94"Platform company" means multi-chip in this case? Seems logical since it's becoming impractical to cram so many transistors on a single die.
no it means rent seeking.
imagine aws if they also sold all computers in the world, now you can only rent from them
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#95What is FP4, 4 bit floating point? If so, the comparison graph [0] with 30x above Hopper was a bit misleading. [0] https://youtu.be/Y2F8yisiS6E?t=4698
That's right.There was mention of a precision aware transformer engine which might make it easier to use fp4, but it's not 30x faster in a like for like way. This shouldn't be surprising since it's more or less two hoppers next to one another on a slightly improved process node. 2.5x seems more likely in cases where you don't exploit a new feature like that or the increased memory.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#96Earlier quoted context omitted.
This seems as good a place as any to be Corrected by the Internet, so... correct me if I'm wrong. Making a graphics chip that is as good as Nvidia: Very difficult. Huge moat, huge effort, lots of barriers, lots of APIs, lot of experience, lots of decades of experience to overcome. Making something that can run a NN: Much, much easier. I'd guess, start-up level feasible. The math is much simpler. There's a lot of it,…
CUDA is a big reason for their moat. And that's not something you can build in a couple of years no matter how money you can throw on it. Without CUDA you have a chip that runs on premise without anyone having a clue how good that is which is supposedly what Google does. Your only offering is cloud services. As big as this is, corporations would want to build their own datacenters.
Sure, CUDA has a lot of highly optimized utilities baked-in (CUDNN and the likes) and maybe more importantly, implementors have a lot of experience with it but afaict everyone is working on their own HAL/compiler and not using CUDA directly to implement the actual models. It's part of the HAL/framework. You can probably port any of these frameworks to a new hardware platform with a few man-years worth of work imo if you can spare the manpower.
I think nobody had the time to port any of these architectures away from CUDA because: * the leaders want to maintain their lead and everyone needs to catch up asap so no time to waste, * and progress was _super_ fast so doubly no time to waste, * there was/is plenty of money that buys some perceived value in maintaining the lead or catching up.
But imo: 1. progress has slowed a bit, maybe there's time to explore alternatives, 2. nvidia GPUs are pretty hard to come by, switching vendors may actually be a competitive advantage (if performance/price pans out and you can actually buy the hardware now as opposed to later).
In terms of ML "compilers"/frameworks, afaik there's:
* Google JAX/Tensorflow XLA/MLIR, * OpenAI Triton, * Meta Glow, * Apple PyTorch+Metal fork.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#97Time to sell. When they start becoming a platform, it means they have nothing more concrete in the near future. Sell now and buy again later once the price corrects.
They still have to announce you can send an email through their platform.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#98Earlier quoted context omitted.
Canonical is a ripe target. Canonical has been trying to grow Ubuntu and other tools in the enterprise world for the last few years without significant success, and much of the Nvidia devkit stuff is built around Ubuntu.
Please do not give them this idea. Ubuntu is actually a pretty great daily driver desktop Linux, and I'd hate for that to lose priority and disappear. I'm not a fan of what happened to the Red Hat ecosystem for exactly the same reasons.
Tbh, Ubuntu’s only pull is the support and breadth of users. As a desktop, it’s let down by Unity, which IMHO is basically a port of Windows 8 tablet UI.
If they defaulted back to a menu and taskbar-based WM, it might actually be more approachable to users who are more familiar with macOS and Windows.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#99Earlier quoted context omitted.
If you are a true believer that AI is not a craze, then the stock can only go up from here. If you think there is a chance that everyone gets bored of AI and moves on to some other fad that is not in Nvidia’s wheelhouse, then it’s probably down from here. I’m staying out of this bet: don’t have the stomach for it.
There's another case for pessimism as well: cost. It's possible that many AI applications aren't worth the money required for the extra compute. AI-enhanced search comes to mind here: how is Microsoft going to monetize users of Copilot in Bing to justify the extra cost? Right now a lot of this stuff is heavily subsidized by VCs or the MSFTs of the world, but when it comes time to make a profit we'll see what actually…
This seems true as far as incentives go. But how much of that cost driver will be due to efficiencies driven by companies like NVIDIA? They seem well poised to benefit from a lot of the increased (non-hype) use of AI. Seems like we spent a decade or more of stalled CPU performance gains chasing better energy efficiency in the data center, same story could play out here.
Re: Nvidia CEO Jensen Huang announces new AI chips: ‘We need bigger GPUs’
#100Stock unchanged in afterhours. A lot of people were hoping for a big pop on some big development.
guy is messing it up bigtime and in real-time as well. sheesh. none of his jokes are landing. “we had 2 customers. we have more now”. long pause. screen behind him covered with logos of all his customers. pause. pause. finally applause. ok on to the next tidbit. whole conference has been proceeding like this now. look if you invite cramer and the wall street crowd, you should throw in some dollar figures. like - who…
Why would a Gen z kid care about this conference?
It's not a hipster event.
And the pauses are clearly his style of presenting. Having a artificial pause to tell the audience it would be a good time to react.
You clearly don't like it, I don't mind it.
But just to be clear: we see peak human ingenuity. This right now will be history on how we as humans build AGI, full human robots etc (in case we don't nuke ourselves).
You can read the conference results easily at any it news portal.
There is no requirement from Nvidia entertaining you or your kid.