Live data from Hacker News

AWS vs. GCP reliability is wildly different

freeman.vc

111–120 of 245 posts

Re: AWS vs. GCP reliability is wildly different

#111
post #73

Earlier quoted context omitted.

In reality it sync the memory first to the other host and only pause the vm when the last state sync is small enough to be so quick the pause is barely measurable.

When its transferring the state to the target, how does it handle memory updates that are happening at that time? Is the programs execution paused at that point?

No, they keep track of dirty pages.

Re: AWS vs. GCP reliability is wildly different

#112
post #14

I wouldn't call this reliability, which already has a loaded definition in the cloud world, and instead something along time-to-start or latency or something.

It is though based on a specific definition. If X doesn't do Y based on Z metric with a large standard deviation and doesn't meet spec limits, it is not reliable as per the predefined tolerance T.

  X = Compute intances
  Y = Launch
  Z = Time to launch
  T = LSL (N/A), USL (10s), Std Dev (2s)
Where LSL is lower spec limit, USL is upper spec limit. LSL is N/A since we don't care if the instance launches instantly (0 seconds).

You can define T as per your requirements. Here we are ignoring the accuracy of the clock that measures time, assuming that the measurement device is infinitely accurate.

If your criteria is to, say for example, define reliability as how fast it shuts down, then this article isn't relevant. Article is pretty narrow in testing reliability, they only care about launch time.

Re: AWS vs. GCP reliability is wildly different

#113
post #73

Earlier quoted context omitted.

In reality it sync the memory first to the other host and only pause the vm when the last state sync is small enough to be so quick the pause is barely measurable.

When its transferring the state to the target, how does it handle memory updates that are happening at that time? Is the programs execution paused at that point?

No, but the memory accesses have hooks that say "This memory was written". Then, program execution is paused, and just the sections of memory that were written are copied again.

This has memory performance characteristics - I ran a benchmark of memory read/write speed while this was happening once. It more than halved memory speed for the 30s or so it took from migration started to migration complete. The pause, too, was much longer.

Re: AWS vs. GCP reliability is wildly different

#114

Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'm starting a Boeing 747 instead of a VM. However, there's one aspect where GCP is a clear winner on the reliability front. They auto-migrate instances transparently and with close to zero impact to workloads – I want to say zero impact but it's not technically…

> Azure feels like I'm starting a Boeing 747 instead of a VM.

Huh... interesting, this has not been my experience with Azure VM launch times. I'm usually surprised how quickly they pop up.

Re: AWS vs. GCP reliability is wildly different

#115
post #71

Earlier quoted context omitted.

Well, if your system elastically uses GPU compute and needs to be able to spin up, run compute on a GPU, and spin down in a predictable amount of time to provide reasonable UX, launch time would definitely be a factor in terms of customer-perceived reliability.

Sure but not anywhere remotely near clearing the bar to simply calling that “reliability”.

When I think “reliability” I think “does it perform the act consistently?”

Consistently slow is still reliability.

Re: AWS vs. GCP reliability is wildly different

#116
Anecdotally I tend to agree with the author. But this really isn't a great way of comparing cloud services.

The fundamental problem with cloud reliability is that it depends on a lot of stuff that's out of your control, that you have no visibility into. I have had services running happily on AWS with no errors, and the next month without changing anything they fail all the time.

Why? Well, we look into it and it turns out AWS changed something behind the scenes. There's a different underlying hardware behind the instance, or some resource started being in high demand because of some other customers.

So, I completely believe that at the time of this test, this particular API was performing a lot better on AWS than on GCP. But I wouldn't count on it still performing this way a month later. Cloud services aren't like a piece of dedicated hardware where you test it one month, and then the next month it behaves roughly the same. They are changing a lot of stuff that you can't see.

Re: AWS vs. GCP reliability is wildly different

#117

Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'm starting a Boeing 747 instead of a VM. However, there's one aspect where GCP is a clear winner on the reliability front. They auto-migrate instances transparently and with close to zero impact to workloads – I want to say zero impact but it's not technically…

> Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent.

FWIW this article is saying the opposite--it's AWS that beats GCP in startup speed.

Re: AWS vs. GCP reliability is wildly different

#118

this doesn't really seem like a fair comparison, nor is it a measure of "reliability".

It seems entirely fair to me, but the term "reliability" has a few different angles. This time it's not about working or not working, but the ability to auto-scale by invoking resources on the spot, which can be a very real requirement.

Re: AWS vs. GCP reliability is wildly different

#119

Earlier quoted context omitted.

If not a 4xx, what should they return for instance not available?

503 service unavailable?

It's not the service that's unavailable. The resource isn't available. The service is running just fine.

Re: AWS vs. GCP reliability is wildly different

#120

Earlier quoted context omitted.

Azure, AWS and GCP all have live migration. VMWare has it too.

Not really. Or at least not in the same league. AWS doesn't have live migration at all. You have to stop/start. Azure technically does, but it doesn't always work(they say 90%). 30 seconds is a long time. VMWare has live migration (and seems to be the closest to what GCP does) but it is still an inferior user experience. This is the key thing you are missing – GCP not only has live migration, but it is completely tra…

VMware's live migration is totally seamless, so I don't know what you mean by "inferior user experience". You typically see less than a second of packet loss, and a small performance hit for about a minute while the memory is "swapped" across to the new machine. Similarly, VMware has had live storage migration for years.

VMware is lightyears ahead of the big clouds, but unfortunately they "missed the boat" on the public cloud, despite having superior foundational technology.

For example:

- A typical vSphere cluster would use live migration to balance workloads dynamically. You don't notice this as an end user, but it allows them to bin-pack workloads up above 80% CPU utilisation in my experience with good results. (Especially if you allocate priorities, min/max limits, etc...)

- You can version-upgrade a vSphere cluster live. This includes rolling hypervisor kernel upgrades and live disk format changes. The upgrade wizard is a fantastic thing that asks only for the cluster controller name and login details! Click "OK" and watch the progress bar.

- Flexible keep-apart and keep-together rules that can updated at any time, and will take effect via live migration. This is sort-of like the Kubernetes "control loops", but the migrations are live and memory-preserving instead of stop-start like with containers.

- Online changes to virtual hardware, including adding not just NICs and disks, but also CPU and memory!

- Thin-provisioned disks, and memory deduplication for efficiencies approaching that of containerisation.

- Flexible snapshots, including the ability for "thin provisioned" virtual machines to share a base snapshot. This is often used for virtual desktops or terminal services, and again this approaches containerisation in terms of cloning speed and storage efficiency.

In other words, VMware had all of the pieces, and just... didn't... use it to make a public cloud. We could have had "cloud.vmware.com" or whatever 15 years ago, but they decided to slowly jack up the price on their enterprise customers instead.

For comparison, in Azure: You can't add a VM to an availability set (keep apart rule) or remove the VM from it without a stop-start cycle. You can't make most changes (SKU, etc...) to a VM in an availability set without turning off every machine in the same AS! This is just one example of many where the public cloud has a "checkbox" availability feature that actually decreases availability. For a long time, changing an IP address in AWS required the VM to be basically blown away and recreated. That brought back memories of the Windows NT 4 days in 1990s when an IP change required a reboot cycle.

Post reply on HN