Earlier quoted context omitted.
Or it's time to go back to one's own infrastructure and take one's own destiny into one's own hands, along with the responsibility.
I can't remember ever seeing this work out well lol. Happy to be proven wrong one day, though.
Your nines are not my nines
111–120 of 135 posts
Re: Your nines are not my nines
#112Earlier quoted context omitted.
Was it like an list of predefined values? Where I work they do root cause analysis for everything, but with freeform answers so what you describe might be different from what I'm used to. In general, I'm so used to RCA and layered mitigations (what one of our greybeards calls "belt and suspenders") that I don't know how quality happens without it. I'm a convert to the idea that if you can't fix a problem directly, th…
Unfortunately this was probably 20 years ago. I know it was a list of predefined values in a drop-down, but I'm not sure if there was a other/write-in field. The gist was that the causes were appropriate and educational. Folks couldn't choose "user is an idiot", instead having to choose "the interface was confusing".
Re: Your nines are not my nines
#113Earlier quoted context omitted.
Maybe it's time for a consumer watchdog group to step in and do their own reporting for services like this. Like https://www.isitdownrightnow.com/ but with sharper teeth. I'm not sure how much time I have to participate but I wouldn't mind chipping in a bit on a co-op in this space. But it might be easier to convince Is it Down Right Now to grow some fangs, or socialize the idea that it does (perception counts for a…
Or it's time to go back to one's own infrastructure and take one's own destiny into one's own hands, along with the responsibility.
But I think I'm going to be waiting a while.
Re: Your nines are not my nines
#114Re: Your nines are not my nines
#115Earlier quoted context omitted.
You may have heard of a company called Amazon.
What thread do you think you are replying to?
Amazon did this. It went pretty well. So well they decided to sell the results of the expertise.
Re: Your nines are not my nines
#116We've talked about these classes of problem from time to time, a lot of ideas have been put forward but what's the solution? If I have a multi-tenant system, and no one customer is dominant (always causes problems IMO), my 'biggest customer' might only be 4% of my traffic. There are a million things that can go wrong that make this customer's experiences different from everyone else's, from getting my sharding soluti…
The issue is that percentiles are a very crude tool. This isn’t just a multitenancy problem, it can manifest in any multiuser system. If 0.5% of your traffic comes from New Zealand, a DNS issue affecting your CDN routing that causes all NZ traffic to time out won’t affect your 99th percentile loadtime graph at all. Essentially, percentiles are useless for discovering problems that have a strong effect on a small port…
Re: Your nines are not my nines
#117This rings so true it hurts. At a very large, very blue, company I recall a time trying to explain to an account manager that I wanted to write the SLAs in terms of my footprint, which is to say, given the resources you have allocated to my account, lets set some SLAs like "latency from any node to any node", "latency from any node to the primary internet", "latency from any node to the secondary internet", "availabi…
Would you be willing to pay more to have the SLAs behave like that, and if so, how much more? Genuinely curious, maybe there's a market for "cloud but better SLAs".
For example, Amazon will give you a 30% refund "for the individual Included Service in the affected AWS region for the monthly billing cycle in which the Unavailability occurred" if availability during a month drops to, but not below, 95% (that's a 1.5 day downtime).
That means that if your service goes 100% down because EC2 was completely broken in a region for 1.5 days, you get a refund of 9 days worth of EC2 (compute) charges, but not the associated EBS (disk) or S3 (storage) or other charges.
And "unavailability" counts only if at least two availability zones at the same time are completely down. And then you have to request the credit in a very specific format.
Google and Azure look extremely similar.
Are better SLAs typically negotiated? Because based on this, it seems like the only thing keeping cloud providers reliably above their SLAs is the fear of losing current and future customers, not the SLAs themselves. In other words, the SLAs are completely meaningless.
Re: Your nines are not my nines
#118Earlier quoted context omitted.
You might want to have a look at some Google Cloud SLAs[1]. They are generally calculated based on actual performance on a particular customer's RPC traffic, down to individual RPCs (at least on Google Cloud Storage, where I work). Read through the agreements to the definition of "Error Rate", and I think you'll find the terms you're looking for. You're welcome! :) The root post does raise an important issue, though…
How are Google Cloud SLAs valuable when the service regularly has multi-hour (IIRC, > 5 hours!) (sometimes global) outages? I pose that Google is not presently in a good position to highlight as a role model or case-study for demonstration of effective cloud provider SLAs.
Re: Your nines are not my nines
#119Earlier quoted context omitted.
How are Google Cloud SLAs valuable when the service regularly has multi-hour (IIRC, > 5 hours!) (sometimes global) outages? I pose that Google is not presently in a good position to highlight as a role model or case-study for demonstration of effective cloud provider SLAs.
>How are Google Cloud SLAs valuable when the service regularly has multi-hour (IIRC, > 5 hours!) (sometimes global) outages? You can sue them? (if that's not forbidden by the contract).
Re: Your nines are not my nines
#120Earlier quoted context omitted.
To get a proper SLA you need to pay for SLA. What SLA google, azure, aws have is useless, some service credit proportional to the outage duration. Totally nothing to cover lost profits or direct damage. For the last gcloud outage, i think you have to talk to people and APPLY for a credit, obviously very few did that https://news.ycombinator.com/item?id=20078296
Working with enterprises that actually have individual SLA's with one of the clouds you mentioned, this is not true. You can handle out your own SLA's with certain providers, and not just get "credits." These enterprises have mature enough monitoring solutions to be able to prove to the provider that they didn't hold up on their end. Besides that, every half-way capable solution architect wouldn't move "system critic…
amazon ec2 clearly states their sla gives you credits https://aws.amazon.com/compute/sla/
azure compute clearly states they give you credits https://azure.microsoft.com/en-us/support/legal/sla/virtual-...
wanna better sla - pay up, like i said in the beginning. as the cost of sla is proportional to payout that works like an insurance, not like coercive measure to increase reliability.