Tesla turns on 10k-node Nvidia H100 Cluster

Post by nickthegreek » Sat, Nov 25, 2023, 9:16 PM UTC

s1gnp0st wrote: post #129

nickthegreek wrote: post #33

Earlier quoted context omitted.

All of his other false or misleading statements over the last 10 years.

When we've dealt with the oil companies, the chemical manufacturers dumping PFAS into our kids, and the industrial war machine, maybe then we can start complaining about the guy biting off more than he can chew trying to be constructive. Until then, all of you sound vicious, bitter, and hypocritical.

Post by _zoltan_ » Tue, Nov 28, 2023, 8:18 AM UTC

latchkey wrote: post #121

_zoltan_ wrote: post #113

Earlier quoted context omitted.

H100 based DGX/HGX doesn't use 800 Gbit (it doesn't have the PCI-e bw), it's using 400 per GPU.

I was talking about between nodes. We're planning on bonding 2x400G NICs to get that 800G between nodes. That said, latest 4th gen nvlink is 900G... https://www.nvidia.com/en-us/data-center/nvlink/ But unless you're sleeping with Jensen, you're not going to see it for 52 weeks+ after you order it.

Post by latchkey » Tue, Nov 28, 2023, 6:58 PM UTC

_zoltan_ wrote: post #133

latchkey wrote: post #121

Earlier quoted context omitted.

I was talking about between nodes. We're planning on bonding 2x400G NICs to get that 800G between nodes. That said, latest 4th gen nvlink is 900G... https://www.nvidia.com/en-us/data-center/nvlink/ But unless you're sleeping with Jensen, you're not going to see it for 52 weeks+ after you order it.

between the GPUs you already have 3.2Tbit/s, plus the 2x400 separately. Pretty sweet. Our lead time hasn't been horrible actually, but I work for a pretty big corp