Earlier quoted context omitted.
Care to name names? I've been looking for a small, cheap failover for a moderately low traffic app.
In the US I use Hivelocity. If you want cheapest possible, Hetzner/OVH have deals you can get for _cheap._
Google Cloud Is Down
561–570 of 630 posts
Re: Google Cloud Is Down
#562Earlier quoted context omitted.
It's not so much AWS vs. in-house. But AWS (or GCP/DO/etc.) vs. multi/hybrid solutions. The latter of which would presumably have lower downtime.
I don't see why multi/hybrid would have lower downtime. All cloud providers as far as I know, though I know mostly of AWS, already have their services in multiple data-centers and their endpoints in multiple regions. So if you make yourself use more then one of their AZs and Region, you would be just as multi as with your own data center.
Although I guess depending on how your own infrastructure is setup, even a multi cloud provider setup won't save you from a network outage like the current Google cloud one.
Re: Google Cloud Is Down
#563Earlier quoted context omitted.
Multi-cloud for those times when you really need that level of availability and can afford it.
It's not even about being able to afford it. Some things just don't lend themselves to hot failover. If your data throughput is high, it may not be feasible or possible to stream a redundant copy to a data center outside the network.
Re: Google Cloud Is Down
#564Earlier quoted context omitted.
Or you wait for Google or you are frantically trying to move everything you got to AWS.
If you wait, you get back to 100% with no effort or stress on your part. If you try to be heroic, you get back to 100% with a bunch of wasted effort and stress on your part. Because it will be fixed by Google, regardless of what you do or don't do. After the incident is over would be the time to consider alternatives.
The other case is really soft failures for multi-region companies. We degrade gracefully, but once that happens, the question becomes what other stuff can you bring back online. For example, this outage did not impact our infrastructure in GCP Frankfurt, however, it prevented internal traffic in GCP from reaching AWS in Virginia because we peer with GCP there. Also couldn't access the Google cloud API to fall back to VPN over public internet. In other cases, you might realize that your failover works, but timeouts are tuned poorly under the specific circumstances, or that disabling some feature brings the remainder of the product back online.
Additionally, you have people on standby to get everything back in order as soon as possible when the provider recover. Also, you may need to bring more of your support team online to deal with increased support calls during the outage.
Re: Google Cloud Is Down
#565You know this reminds me of a bad taste that Google Sales team left when I asked for some of my billing that I was unaware of running after following a quickstart guide. AWS refunded me in the first reply on the same day! GCP sales rep just copy pasted a link to a self support survey that essentially told me, after a series of YES or NO questions that they can't refund me. So why not just tell your customers like it…
I think it's weird to say you get credit in dollars and then not be able to spend it on everything. That's not how money works. But that's the way hosting providers work and afaik it's quite well known. Especially with a large sum of "free money", even if it's not well known, it was on you to check any small print.
Re: Google Cloud Is Down
#566Earlier quoted context omitted.
In the US I use Hivelocity. If you want cheapest possible, Hetzner/OVH have deals you can get for _cheap._
I've a question that always stopped me going that route, what happens when a disk or other hardware fails on these servers? beyond data loss I mean, like physically what happens who carries out the repair how long does it takes
Re: Google Cloud Is Down
#567Earlier quoted context omitted.
We don’t usually discuss the frequency of unplanned failovers, but I will tell you that we do a planned failover at least every two weeks. The team also uses traffic shaping to perform whole system load tests with production traffic, which happens quarterly.
Do you do any chaos testing? Seems like it would slot right in, there.
Re: Google Cloud Is Down
#568This is your yearly reminder to resist centralization of the internet.
Re: Google Cloud Is Down
#569Earlier quoted context omitted.
I've a question that always stopped me going that route, what happens when a disk or other hardware fails on these servers? beyond data loss I mean, like physically what happens who carries out the repair how long does it takes
Most bare metal providers nowadays contact you just like AWS and say "hey your hardware is failing get a new box.". Unless it's something exotic it's usually not long for setup time, and in some cases just like a VM it's online in a minute or two.
Re: Google Cloud Is Down
#570Disclosure: I work on Google Cloud (but disclaimer, I'm on vacation and so not much use to you!). We're having what appears to be a serious networking outage. It's disrupting everything, including unfortunately the tooling we usually use to communicate across the company about outages. There are backup plans, of course, but I wanted to at least come here to say: you're not crazy, nothing is lost (to those concerns do…