So how do they get this data to the GPUs now...? Just run it over the public internet to the datacenter?
They can rent a dark fiber for themselves for that distance, and it'll be cheap. However, as they noted they use 100gbps capacity from their ISP.
Re: Building the heap: racking 30 petabytes of hard drives for pretraining
#11We want to get darkfiber from the datacenter to the office. I love 100Gbps