Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS vs. GCP reliability is wildly different

freeman.vc

Post reply on HN
Print view View original

231–240 of 245 posts

Re: AWS vs. GCP reliability is wildly different

#231

Post by akramer »

stevenguh wrote: post #193
jsolson wrote: post #184

Earlier quoted context omitted.

Idempotency. You're inserting a VM with a specific name. If you try to create the same resource twice, the GCE control plane reports that as a conflict. What they're doing here would be roughly equivalent to supplying the time to the AWS RunInstances API as an idempotency token. (I work on GCE, and asked an industry friend at AWS about how they guarantee idempotency for RunInstances).

GCP control plane is generally not idempotent. When trying to create the same resource twice, all request should report the same status instead one failing, one succeeding. In AWS, their APIs allow you to supply a client token if the API is not idempotent by default. See https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_I... .

The GCE API can be idempotent if you'd like. Fill out the requestId field with the same UUID in multiple instances.insert calls (or other mutation calls) and you will receive the same operation Id back in response.

Disclaimer: I work on GCE.

Re: AWS vs. GCP reliability is wildly different

#232

Post by tl_donson »

iLoveOncall wrote: post #81
dark-star wrote: post #13

I wonder why someone would equate "instance launch time" with "reliability"... I won't go as far as calling it "clickbait" but wouldn't some other noun ("startup performance is wildly different") have made more sense?

It is clickbait, the real title should be "AWS vs. GCP on-demand provisioning of GPU resources performance is wildly different". That said, while I agree that launch time and provisioning error rate are not sufficient to define reliability, they are definitely a part of it.

“ AWS vs. GCP on-demand provisioning of GPU resources performance is wildly different”

yeah i guess it does make sense that one didn’t win the a/b test

Re: AWS vs. GCP reliability is wildly different

#233

Post by philliphaydon »

jsolson wrote: post #227
philliphaydon wrote: post #213

Earlier quoted context omitted.

Sounds like AWS got it right.

You're entitled to that takeaway, but I disagree. I believe GCP's tendency to use caller-supplied names for resources is one of the single best features of the platform, particularly when compared against AWS's random hex identifiers. Note that whether this creates collisions is entirely under the customer's control. There's no requirement for global uniqueness, just a requirement that you not try to create two VMs w…

With GCE can you create 10 instances or do you need to create all 10 individually?

Re: AWS vs. GCP reliability is wildly different

#234

Post by _msw_ »

darkwater wrote: post #220
_msw_ wrote: post #138

Earlier quoted context omitted.

Disclosure: I work for Amazon, and in the past I worked directly on EC2. From the FAQ: https://aws.amazon.com/ec2/faqs/ Q: How does EC2 perform maintenance? AWS regularly performs routine hardware, power, and network maintenance with minimal disruption across all EC2 instance types. To achieve this we employ a combination of tools and methods across the entire AWS Global infrastructure, such as redundant and concurre…

> AWS regularly performs routine hardware, power, and network maintenance with minimal disruption across all EC2 instance types. To achieve this we employ a combination of tools and methods across the entire AWS Global infrastructure, such as redundant and concurrently maintainable systems, as well as live system updates and migration. And yet, I keep getting almost every weeks emails like this: "EC2 has detected deg…

Live migration can't be used to address every type of maintenance or underlying fault in a non-disruptive way.

Re: AWS vs. GCP reliability is wildly different

#235

Post by jsolson »

philliphaydon wrote: post #233
jsolson wrote: post #227

Earlier quoted context omitted.

You're entitled to that takeaway, but I disagree. I believe GCP's tendency to use caller-supplied names for resources is one of the single best features of the platform, particularly when compared against AWS's random hex identifiers. Note that whether this creates collisions is entirely under the customer's control. There's no requirement for global uniqueness, just a requirement that you not try to create two VMs w…

With GCE can you create 10 instances or do you need to create all 10 individually?

As far as I know, the `instances.insert` API only allows individual VMs, although the CLI can issue a bulk set of API calls[0], and MIGs (see below) allow you to request many identical VMs with a single API call if that's for some reason important.

You can also batch API calls[1], which also gives you a response for each VM in the batch while allowing for a single HTTP request/response.

That said, if you want to create a set of effectively identical VMs all matching a template (i.e., cattle not pets), though, or you want to issue a single API call, we'd generally point you to managed instance groups[2] (which can be manually or automatically scaled up or down) wherein you supply an instance template and an instance count. The MIG is named (like nearly all GCP resources), as are the instances, with a name derived from the MIG name. After creation you can also have the group abandon the instances and then delete the group if you really wanted a bunch of unmanaged VMs created through a single API call, although I'll admit I can't think of a use-case for this (the abandon API is generally intended for pulling VMs out of a group for debugging purposes or similar).

For cases where for whatever reason you don't want a MIG (e.g., because your VMs don't share a common template). You can still group those together for monitoring purposes[3], although it's an after-creation operation.

The MIG approach sets a _goal_ for the instance count and will attempt to achieve (and maintain) that goal even in the face of limited machine stock, hardware failures, etc. The top-level API will reject (stock-out) in the event that we're out of capacity, or in the batch/bulk case will start rejecting once we run out of capacity. I don't know how AWS's RunInstances behaves if it can only partially fulfill a request in a given zone.

[0]: https://cloud.google.com/compute/docs/instances/multiple/cre...

[1]: https://cloud.google.com/compute/docs/api/how-tos/batch

[2]: https://cloud.google.com/compute/docs/instance-groups

[3]: https://cloud.google.com/compute/docs/instance-groups/creati...

Re: AWS vs. GCP reliability is wildly different

#236

Post by jsolson »

akramer wrote: post #231
stevenguh wrote: post #193

Earlier quoted context omitted.

GCP control plane is generally not idempotent. When trying to create the same resource twice, all request should report the same status instead one failing, one succeeding. In AWS, their APIs allow you to supply a client token if the API is not idempotent by default. See https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Run_I... .

The GCE API can be idempotent if you'd like. Fill out the requestId field with the same UUID in multiple instances.insert calls (or other mutation calls) and you will receive the same operation Id back in response. Disclaimer: I work on GCE.

Today I learned! I'll admit I didn't know this functionality existed, and I've instead had used instances.insert following by querying the VM resource.

This is nicer!

Re: AWS vs. GCP reliability is wildly different

#238

Post by w0m »

sofixa wrote: post #198
jiggawatts wrote: post #120

Earlier quoted context omitted.

VMware's live migration is totally seamless, so I don't know what you mean by "inferior user experience". You typically see less than a second of packet loss, and a small performance hit for about a minute while the memory is "swapped" across to the new machine. Similarly, VMware has had live storage migration for years. VMware is lightyears ahead of the big clouds, but unfortunately they "missed the boat" on the pub…

So, i used to be a part time vSphere admin, worked with many others, and had to automate the hell out of it to deal as little as possible with that dumpster fire. No, VMware didn't miss the boat, vCloud Air was announced in 2009 and made generally available in 2013. Roughly same timelines as Azure and GCP, slightly trailing AWS, and those were the early days, where the public cloud was still exotic. And VMware had th…

> many of those features were either behind a Flash UI(FFS)

My introduction to the industry. The memories.

Re: AWS vs. GCP reliability is wildly different

#239

Post by jrumbut »

flutas wrote: post #221
jrumbut wrote: post #175

Earlier quoted context omitted.

> (3) `time.time()` is not monotonic. I just winced in pain thinking of the ways that can bite you. I guess in a cloud/virtualized environment with many short lived instances it isn't even that obscure an issue to run into. A nice discussion on Stack Overflow: https://stackoverflow.com/questions/64497035/is-time-from-ti...

> I just winced in pain thinking of the ways that can bite you. Something similar caused my favorite bug so far to track down. We were seeing odd spikes in our video playback analytics of some devices watching multiple years worth of video in System.currenTimeMillis() in Java isn't monotonic either is my short answer for what was causing it. Tracking down _what_ was causing it was even more fun though. Devices (phone…

That's a bad day at the office when you have to go and say "hey remember all that data we painstakingly collected and maybe even billed clients for?"

Re: AWS vs. GCP reliability is wildly different

#240

Post by vineyardmike »

smashed wrote: post #174
sitharus wrote: post #131

Earlier quoted context omitted.

This is a very good point - AWS uses tags to give instances a friendly name, so the name does not have to be unique. The same logic would not fail on AWS.

Which makes 2000% sense. Why would any tenant supplied data affect anything whatsoever? As a tenant, unless you are clashing with another resource under your own name, I don't see the point of failing. aws S3 would be an exception, where they make that limitation on globally unique bucket name very clear.

> unless you are clashing with another resource under your own name, I don't see the point of failing.

Is that not the conclusion? The tester was clashing with their own names?

Post reply on HN