[deleted]
Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
101–110 of 182 posts
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#102People want to rent the pricey NVIDIA DGX H100. So Google just put it in their DC, letting customers pay its full price it every ~3 months; plus they don't have to operate it, which is win (or is it ?).
I also want to run my job on 5 DGXs for a month, not 1 DGX for 5 months.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#103Kinda feels like the main thing google launches is waiting lists.
Gmail was a waitlist or invite only for many years. And that must’ve been their most successful product launch since search
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#104AMD should be gifting their GPUs by the dozens to the most prolific Open Source contributors if they want a piece of the cake. Their lack of access to CUDA is really harming them badly.
It's more than just that: for the money, their consumer GPUs don't compete in compute tasks (especially inference/training) and their Linux compute drivers are a pile of steaming garbage on consumer hardware. It's really interesting/depressing to watch as they've done a nice job of supplying good open source graphics drivers. They really seem to be lacking something at a leadership level in terms of understanding GPU…
Whereas NVIDIA is basically just “we’re an AI company now”.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#105Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second para…
Companies never announce change of direction like you seem to think. There is no positive outcome in doing so. Instead they announce the new thing and promise to continue to support the old thing and then just don’t.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#106Earlier quoted context omitted.
We need to stop feeding the advertising machine. That'll starve Google and other advertising parasites. First step in doing that is opening up the Android ecosystem and legislating Google's hands out of that pie. I can't even so much as shit on an android phone without requiring a valid Google account. /crass joke
I think the name in question is Nvidia, not Google.
This was launched by Google, hence my comment. But yeah, guess it's just as likely the other comment was about Nvidia.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#107So this is why Nvidia isn't lowering the price on the GPUs despite them sitting on the shelves and not selling. They make enough money from customers in the data center and supercomputer businesses that gaming is just a small market.
H100s are not sitting on shelves, even at the 35kUSD price sticker. Consumer GPUs, probably yes. Even for datacenter compute workloads that would not go for H100, the L40 is supposedly 3xA40 in FP32 FLOPs but still on the same memory bandwidth, so who knows what kind of performance you'll get whenever you can get your OEM to build you one......
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#108Earlier quoted context omitted.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
I think in this case they just know the demand is RED HOT and they don't have nearly the supply to go around. I don't think it's really the typical new product concerns on this one (product-market fit, are we covering use cases, are there technical problems, etc.). They know people want this and would rather have it right this second, problems and all, than wait for a slow rollout; Google just doesn't have the supply…
Then sell it for more.
Just give me a price. Or let me bid on it.
Or, don't announce it like it's launched until it's usable.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#109People want to rent the pricey NVIDIA DGX H100. So Google just put it in their DC, letting customers pay its full price it every ~3 months; plus they don't have to operate it, which is win (or is it ?).
they have to pay for the power and the staff to set them up/manage it
That's generally the cloud in a nutshell, they price accordingly
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#110Earlier quoted context omitted.
Sure, but look at Bard it was on wait-list for what 3 months? Now it's available in 180 countries... for free. Not everything gets launched because sometimes they find out in that testing period that they got it wrong.
After the big hype of it yes. And the models are not really available, they've got a little playground for some unspecified model.
There are models available in GCP under Vertex AI category, I'm using the API to access them.