41–50 of 245 posts
Earlier quoted context omitted.
GCP also had 84 errors compared to 1 for AWS
Maybe 1 reported. Not saying aws reliability is bad, but the number of various glitches that crop up in various aws services and not reflected on their status page is quite high.
> The offerings between the two cloud vendors are also not the same, which might relate to their differing response times. GCP allows you to attach a GPU to an arbitrary VM as a hardware accelerator - you can separately configure quantity of the CPUs as needed. AWS only provisions defined VMs that have GPUs attached - the g4dn.x series of hardware here. Each of these instances are fixed in their CPU allocation, so if…
> why I burned $150 on GPUs
How do you rent 3000 GPUs over a period of weeks for $150? Were they literally requisitioning it and releasing it immediately? Seems like this is quite a unrealistic type of usage pattern and would depend a lot on whether the cloud provider optimises to hand you back the same warm instance you just relinquished.
> GCP allows you to attach a GPU to an arbitrary VM as a hardware accelerator
it's quite fascinating that GCP can do this. GPUs are physical things (!) do they provision every single instance type in the data center with GPUs? That would seem very expensive.
I wonder why someone would equate "instance launch time" with "reliability"... I won't go as far as calling it "clickbait" but wouldn't some other noun ("startup performance is wildly different") have made more sense?
Earlier quoted context omitted.
Dunno about AWS, but GCP uses live migration, and will migrate your VM across physical machines as necessary. The disk volumes are all connected over the network, nothing really depends on the actual physical machine your VM is ran on.
Azure, AWS and GCP all have live migration. VMWare has it too.
AWS normally has machines sitting idle just waiting for you to use. Thats why they can get you going in a couple of seconds. GCP on the other hand fills all machines with background jobs. When you want a machine, they need to terminate a background job to make room for you. That background job has a shutdown grace time. Usually thats 30 seconds. Sometimes, to prevent fragmentation, they actually need to shuffle aroun…
> In total it scaled up about 3,000 T4 GPUs per platform > why I burned $150 on GPUs How do you rent 3000 GPUs over a period of weeks for $150? Were they literally requisitioning it and releasing it immediately? Seems like this is quite a unrealistic type of usage pattern and would depend a lot on whether the cloud provider optimises to hand you back the same warm instance you just relinquished. > GCP allows you to a…
However, live-migration can cause impact to HPC workloads.
Heard from a Googler that the internal infrastructure (Borg) is simply not optimized for quick startup. Launching a new Borg job often takes multiple minutes before the job runs. Not surprising at all.