I'm more interested in what normal folks are running at home. What are your builds?
4090 and 3090 on personal desktop; 4 x 2080Ti in data center
Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
41–50 of 182 posts
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#42Kinda feels like the main thing google launches is waiting lists.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#43Does this mean google just deprecated TPUs? Not surprised.
TPUs do compete with GPUs for ML tasks, so yes, this is evidence that GPUs are winning. The only alternative I could imagine is that TPUs will "win" at supercomputers exclusivity aimed at inference (as opposed to training). Since TPUs excel at inference. The question is how much ML compute is used for inference as opposed to training. Not much, I guess, otherwise something like TPUs would be more popular.
Since you seem knowledgable on this topic, what is it that TPUs do differently than GPUs? Why are they better at inference?
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#44Kinda feels like the main thing google launches is waiting lists.
Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.
It also traps them in a continual cycle of missing the hype wave, and then shutting down the unpopular product a few years later.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#45I'm more interested in what normal folks are running at home. What are your builds?
RTX 3080 on a laptop. 8 GB was more than enough for gaming, but I get out of vram errors quite frequently.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#46I'm more interested in what normal folks are running at home. What are your builds?
FYI rumour has it next round of titan GPUs are supposedly coming with 48GB
Of course there is always something better on horizon, but if you're building soon that may be worth the wait
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#47Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better.
And isn't your second paragraph the obvious reason for why this product (A3) exists? It's something they expect to sell to cloud customers who have an existing GPU-based workflow, and just want to run it as-is as fast/cheap/scalable as possible, without worrying about compatibility, and making sure they can always move the workload to some other cloud provider or on-prem if needed.
It's like suggesting Sony releasing some of their games on the PC means they're deprecating Playstation.
(Maybe there would be more details in the IO talk. Does anyone know which one this announcement is from?)
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#48Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
There’s demand in GCP for H100s so they offer them. I doubt Google itself is a big user.
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#49Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…
[deleted]
Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs
#50So this is why Nvidia isn't lowering the price on the GPUs despite them sitting on the shelves and not selling. They make enough money from customers in the data center and supercomputer businesses that gaming is just a small market.
I bought a desktop in New York City a month ago with a Nvidia RTX 4090 card at Best Buy - 4090 being the most powerful Nvidia card Best Buy had in stock. At that time (a month ago) there were several desktops with this card in stock around the city, and I bought the one I wanted (if I had more time my purchase might have been different). Looking right now - I don't see any unbundled Nvidia RTX 4090 cards for sale at…
Microcenter here in Overland Park, Kansas had at least one of each of the major brands of 4090s available for sale in store last week when I was there. Do people go to Best Buy to buy ultra high end graphics cards? I haven’t bought a graphics card at Best Buy since they used to scam people by putting “pro” at the end of a worse product back in 2003 or so.