Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

Post by IanCal » Sat, May 13, 2023, 5:20 PM UTC

cloudking wrote: post #70

IanCal wrote: post #65

Earlier quoted context omitted.

I get it, but the launches are always about what you can now do then slowly followed with "some partners can register interest". For example for palm/bard this was my experience: "Hey we have this amazing LLM!" "Great, given you are a company can I pay you money above your costs for this service?" "No but you can register for updates about when the wait-list will open " They announced cool features for Google docs as…

Sure, but look at Bard it was on wait-list for what 3 months? Now it's available in 180 countries... for free. Not everything gets launched because sometimes they find out in that testing period that they got it wrong.

Post by smoldesu » Sat, May 13, 2023, 5:23 PM UTC

noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

Post by smoldesu » Sat, May 13, 2023, 5:25 PM UTC

jocaal wrote: post #73

zrm wrote: post #68

Earlier quoted context omitted.

Interestingly the "4th generation Intel Xeon Scalable processors" themselves have up to 2.45 TBps in memory bandwidth, with the 8-socket configuration, or 2 TBps with 2-socket Xeon Max and HBM. If they'd make an 8-socket Xeon Max it would have 8 TBps. Considering that the Xeon Max 9462 is $8000 vs. the H100 going for north of $40,000, that could be interesting.

The throughput these gpu's have make the price pretty competitive, but I think AMD is working on a APU in their instinct lineup. That could be pretty competitive since Nvidia is overcharging for memory and you could just use sticks instead

Post by jsnell » Sat, May 13, 2023, 5:30 PM UTC

onion2k wrote: post #56

jsnell wrote: post #47

Earlier quoted context omitted.

There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better. And isn't your second para…

The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.

Post by lwkl » Sat, May 13, 2023, 5:31 PM UTC

metadat wrote: post #10

jleahy wrote: post #4

Earlier quoted context omitted.

250

Interesting, so what is the compute power of the 1000-node A100 super cluster my team has been allocated at work? I was expecting Google to be much bigger than us.