Pretty crazy. Supporting 1.5mbps video calls for each human on earth? Did I read that right?
Just goes to show how drastic and extraordinary levels of scale can be.
Speed, scale and reliability: 25 years of Google datacenter networking evolution
21–30 of 91 posts
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#22[deleted]
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#23Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#24The most amazing surveillance machine ever …
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#25Awesome Google... Now learn what an availability zone is and stop creating them with firewalls across the same data center.
Oh and make your data centers smaller. Not so big they can be seen in Google Maps. Because otherwise, you will be unable to move those whale sized workloads to an alternative.
https://youtu.be/mDNHK-SzXEM?t=564
https://news.ycombinator.com/item?id=35713001
"Unmasking Google Cloud: How to Determine if a Region Supports Physical Zone Separation" - https://cagataygurturk.medium.com/unmasking-google-cloud-how...
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#26Awesome Google... Now learn what an availability zone is and stop creating them with firewalls across the same data center. Oh and make your data centers smaller. Not so big they can be seen in Google Maps. Because otherwise, you will be unable to move those whale sized workloads to an alternative. https://youtu.be/mDNHK-SzXEM?t=564 https://news.ycombinator.com/item?id=35713001 "Unmasking Google Cloud: How to Determi…
Making a datacenter not visible from Google Maps, at least on most big cities where Google zones are deployed, would mean making them smaller than a car. Or even smaller than a dishwasher.
If I check London (where europe-west2 is kinda located) on Google Maps right now, I can easily discern manhole covers or people. If I check Jakarta (Asia-southeast2) things smaller than a car get confusing, but you can definitely see them.
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#27This mentions Jupiter generations, which I think is about 10-15 years old at this point. It doesn't really talk about what existed before so it's not really 25 years of history here. I want to say "Watchtower" was before Jupiter? but honestly it's been about a decade since I read anything about it. Google's DC networking is interesting because of how deeply integrated it is into the entire software stack. Click on so…
This latest revision of Jupiter is apparently 400G, as is the ConnectX-7, A3 Ultra will have 8 of them!
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#28[flagged]
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#29Does gcp have the worst networking for gpu training though?
Re: Speed, scale and reliability: 25 years of Google datacenter networking evolution
#30It seems all cutting edge datacenters like x.ai Colossus are using Nvidia networking. Now Google is upgrading to Nvidia networking, too. Since Nvidia owns most of the Gpgpu products, they have top notch networking and interconnect, I wonder if they don't have a plan to own all datacenter hardware in the future. Maybe they plan to also release CPUs, motherboards, storage and whatever else is needed.
Nvidia networking is what used to be called Mellanox networking, which was already dominant in datacenters.
Only within supercomputers (including the smaller GPU ones used to train AI). Normal data centers use Cisco or Juniper or similarly.well known Ethernet equipment, and they still do. The Mellanox/Nvidia Infiniband networks are specifically used for supercomputer-like clusters.