We have constant autoscaling issues because of this in GCP - glad someone plotted this - hope people in GCP will pay a bit more attention to this. Thanks to the OP!
AWS vs. GCP reliability is wildly different
101–110 of 245 posts
Re: AWS vs. GCP reliability is wildly different
#102I wonder why someone would equate "instance launch time" with "reliability"... I won't go as far as calling it "clickbait" but wouldn't some other noun ("startup performance is wildly different") have made more sense?
Well, if your system elastically uses GPU compute and needs to be able to spin up, run compute on a GPU, and spin down in a predictable amount of time to provide reasonable UX, launch time would definitely be a factor in terms of customer-perceived reliability.
I'd still consider it as "performance issue", not "reliability issue". There is no service unavailability here. It just takes your system a minute longer until the target GPU capacity is available. Until then it runs on fewer GPU resources, which makes it slower. Hence performance.
The errors might be considered a reliability issue, but then again, errors are a very common thing in large distributed systems, and any orchestrator/autoscaler would just re-try the instance creation and succeed. Again, a performance impact (since it takes longer until your target capacity is reached) but reliability? not really
Re: AWS vs. GCP reliability is wildly different
#103Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'm starting a Boeing 747 instead of a VM.
However, there's one aspect where GCP is a clear winner on the reliability front. They auto-migrate instances transparently and with close to zero impact to workloads – I want to say zero impact but it's not technically zero.
In comparison, in AWS you need to stop/start your instance yourself so that it will move to another hypervisor(depending on the actual issue AWS may do it for you). That definitely has impact on your workloads. We can sometimes architect around it but there's still something to worry about. Given the number of instances we run, we have multiple machines to deal with weekly. We get all these 'scheduled maintenance' events (which sometimes aren't really all that scheduled), with some instance IDs(they don't even bother sending the name tag), and we have to deal with that.
I already thought stop/start was an improvement on tech at the time (Openstack, for example, or even VMWare) just because we don't have to think about hypervisors, we don't have to know, we don't care. We don't have to ask for migrations to be performed, hypervisors are pretty much stateless.
However, on GCP? We had to stop/start instances exactly zero times, out of the thousands we run and have been running for years. We can see auto-migration events when we bother checking the logs. Otherwise, we don't even notice the migration happened.
It's pretty old tech too:
https://cloudplatform.googleblog.com/2015/03/Google-Compute-...
Re: AWS vs. GCP reliability is wildly different
#104Earlier quoted context omitted.
Well, if your system elastically uses GPU compute and needs to be able to spin up, run compute on a GPU, and spin down in a predictable amount of time to provide reasonable UX, launch time would definitely be a factor in terms of customer-perceived reliability.
I'd still consider it as "performance issue", not "reliability issue". There is no service unavailability here. It just takes your system a minute longer until the target GPU capacity is available. Until then it runs on fewer GPU resources, which makes it slower. Hence performance. The errors might be considered a reliability issue, but then again, errors are a very common thing in large distributed systems, and any…
I’d like to see a breakdown of the cost differences. If the costs are nearly equal, why would I not choose the one that has a faster startup time and fewer errors?
Re: AWS vs. GCP reliability is wildly different
#105Is this testing for spot instances? In my limited experience, persistent (on-demand) GCP instances always boot up much faster than AWS EC2 instances.
I noticed that too and it does appear to be using spot instances. I have a feeling if it was ran without you may see much better startup times. Spot instances on GCP are hit and miss and you sort of have to build that into your workflow.
Re: AWS vs. GCP reliability is wildly different
#106Re: AWS vs. GCP reliability is wildly different
#107Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'm starting a Boeing 747 instead of a VM. However, there's one aspect where GCP is a clear winner on the reliability front. They auto-migrate instances transparently and with close to zero impact to workloads – I want to say zero impact but it's not technically…
I always wondered why you couldn't do that on AWS, mainly because I could do it at home with Hyper-V a decade ago.
https://learn.microsoft.com/en-us/previous-versions/windows/...
Re: AWS vs. GCP reliability is wildly different
#108Earlier quoted context omitted.
Sure but not anywhere remotely near clearing the bar to simply calling that “reliability”.
It is not reliably running the machine but reliably getting the machine. Like the article said, The promise of the cloud is that you can easily get machines when you need them the cloud that sometimes does not get you that machine(or does not get you that machine in time) is a less reliable cloud than the one that does.
It’s still performance. If this was “AWE failed to deliver the new machines and GCP delivered”, sure, reliability. But this isn’t that.
The race car that finishes first is not “more reliable” than the one in 10th. They are equally as reliable, having both finished the race. The first place car is simply faster at the task.
Re: AWS vs. GCP reliability is wildly different
#109[deleted]
Re: AWS vs. GCP reliability is wildly different
#110Earlier quoted context omitted.
Dunno about AWS, but GCP uses live migration, and will migrate your VM across physical machines as necessary. The disk volumes are all connected over the network, nothing really depends on the actual physical machine your VM is ran on.
Azure, AWS and GCP all have live migration. VMWare has it too.
Not really. Or at least not in the same league.
AWS doesn't have live migration at all. You have to stop/start.
Azure technically does, but it doesn't always work(they say 90%). 30 seconds is a long time.
VMWare has live migration (and seems to be the closest to what GCP does) but it is still an inferior user experience.
This is the key thing you are missing – GCP not only has live migration, but it is completely transparent. We do not have to initiate migration. GCP does, transparently, 100% of the time. We have never even notice migrations even when we were actively watching those instances. We don't know or care what hypervisors are involved. They even preserve the network connections.
https://cloudplatform.googleblog.com/2015/03/Google-Compute-...