Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

Post by jocaal » Sat, May 13, 2023, 4:47 PM UTC

impulser_ wrote: post #55

lordofgibbons wrote: post #38

Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…

This is for GCP. Google themselves probably still trains on custom hardware but they don't offer their latest and greatest hardware on GCP. Offering more options to customers is always better especially when Nvidia has great market share in this area. This is probably the reason why Microsoft is trying to help AMD catch up so their is more competition. AI GPU prices are insane compared to standard GPU because of the…

Post by jocaal » Sat, May 13, 2023, 4:54 PM UTC

zrm wrote: post #68

binkHN wrote: post #25

Technical detail: > Each A3 supercomputer is packed with 4th generation Intel Xeon Scalable processors backed by 2TB of DDR5-4800 memory. But the real "brains" of the operation come from the eight Nvidia H100 "Hopper" GPUs, which have access to 3.6 TBps of bisectional bandwidth by leveraging NVLink 4.0 and NVSwitch.

Interestingly the "4th generation Intel Xeon Scalable processors" themselves have up to 2.45 TBps in memory bandwidth, with the 8-socket configuration, or 2 TBps with 2-socket Xeon Max and HBM. If they'd make an 8-socket Xeon Max it would have 8 TBps. Considering that the Xeon Max 9462 is $8000 vs. the H100 going for north of $40,000, that could be interesting.

Post by vsareto » Sat, May 13, 2023, 5:04 PM UTC

Post by noogle » Sat, May 13, 2023, 5:09 PM UTC

Post by [deleted] » Sat, May 13, 2023, 5:13 PM UTC

ChuckNorris89 wrote: post #67

cloudking wrote: post #42

Earlier quoted context omitted.

Yep, launching things slowly and testing them before releasing wide. Seems like a good practice when you're introducing a new technology to the world.

Imagine if Apple when launching a new iPhone would first launch to a small country for testing, like Philippines or something, and then slowly expand worldwide. That would drive consumers nuts.