Live data from Hacker News

Google Cloud Is Down

news.ycombinator.com

491–500 of 630 posts

Re: Google Cloud Is Down

#491

Earlier quoted context omitted.

Just like AWS, then. "Some users are experiencing increased error rates" = "Everything has been down for hours"

"Everything is fine, unless you're Carl. There's a massage outage, but only at Carl's house. Sorry, Carl."

I'm also experiencing a massage outage. Please send masseuse.

Re: Google Cloud Is Down

#492

Nest is down too, not surprising given they are part of Google. What I don't understand is why I can't still control my devices over my local network. Why does the system even require access to Google servers?

Let's make the following the tech mantra of the next decade: "the Internet Of Things can never possibly work until the LAN Of Things does".

Re: Google Cloud Is Down

#493

Earlier quoted context omitted.

You obviously don't recall the early years of AWS. Half of internet would go down for hours.

Back when S3 failures would take town Reddit, parts of Twitter .. Netflix survived because they had additional availability zones. I can remember the bigger names started moving more stuff to their own data centers. AWS tries to lock people in to specific services now which makes it really difficult to migrate. It also takes a while before you get to the tipping point where hosting your own is more financially viable…

Netflix actually added the additional AZs because of a prior outage that did take them down.

"After a 2012 storm-related power outage at Amazon during which Netflix suffered through three hours of downtime, a Netflix engineer noted that the company had begun to work with Amazon to eliminate “single points of failure that cause region-wide outages.” They understood it was the company’s responsibility to ensure Netflix was available to entertain their customers no matter what. It would not suffice to blame their cloud provider when someone could not relax and watch a movie at the end of a long day."

https://www.networkworld.com/article/3178076/why-netflix-did...

Re: Google Cloud Is Down

#494
post #370

Earlier quoted context omitted.

How can they possibly guarantee eleven nines? Considering I’ve never heard of this company and they offer such crazy-sounding improvements over the big three, it feels like there should be a catch.

11 9s isn't uncommon. AWS S3 does 11 9s (upto 16 9s with cross region replication?) for data durability, too. AFAIK, AWS published papers about their use of formal methods to ascertain bugs from other parts of the system didn't creep in to affect durability/availability guarantees: https://blog.acolyer.org/2014/11/24/use-of-formal-methods-at... This is a pretty neat and concise read on ObjectStorage in-use at BigTech…

You have to be kidding me. 14 9's is already microseconds a year. Surely below anybody's error bar for whether a service is down or not.

16 9's and aws should easily last as long as the great pyramids without a second worth of outage.

What a joke

Re: Google Cloud Is Down

#495
post #151
post #63

Disclosure: I work on Google Cloud (but disclaimer, I'm on vacation and so not much use to you!). We're having what appears to be a serious networking outage. It's disrupting everything, including unfortunately the tooling we usually use to communicate across the company about outages. There are backup plans, of course, but I wanted to at least come here to say: you're not crazy, nothing is lost (to those concerns do…

To clarify something: this outage doesn’t appear to be global, but it is hitting us particularly hard in parts of the US. So for the folks with working VMs in Mumbai, you’re not crazy. But for everyone with sadness in us-central1, the team is on it.

It seemed to be congestion in the North East US.

Re: Google Cloud Is Down

#496
post #219

Earlier quoted context omitted.

Always in Virginia, because US-east has always been cheaper.

The only regions that are more expensive than us-east-1 in the States are GovCloud and us-west-1 (Bay Area). Both us-west-2 (Oregon) and us-east-2 (Ohio) are priced the same as us-east-1.

I would probably go with US-EAST-2 just because it's isolated from anything except perhaps a freak Tornado and better situated on the eastern US. Latency to/from there should be near optimal for most eastern US/Canada population.

Re: Google Cloud Is Down

#497

Earlier quoted context omitted.

I mean, I care. All else being equal I’m not sure why you wouldn’t want good communication to your customers.

How much cloud spend do you control? That's the reality of how decisions are made.

Many millions of dollars per year. I care about how my providers behave when they have issues, and I can't see why you think it's not at all relevant.

Re: Google Cloud Is Down

#498
post #341

Earlier quoted context omitted.

No, the graphs are highly misleading because they autoscale the y axis to the highest point and the they do very loose string matching to detect errors. If you click through to something not on Google cloud, you see moderately elevated error rates (e.g., Instagram is up by 4x) but if you click through to something actually on Google, you see very highly elevated error rates (e.g., roughly 50000x for Snapchat). If you…

Ah. I was looking at it on phone and it wasn't clear. Thanks for clarifying.

it's no clearer on desktop. that's actually pretty funny that the instagram graph spikes just because people are complaining about snapchat being down.

i always thought downdetector was doing something a bit more clever than just reporting the rate of tweets containing the word "instagram" or "facebook" or something. but apparently not.

Re: Google Cloud Is Down

#499

Earlier quoted context omitted.

"Everything is fine, unless you're Carl. There's a massage outage, but only at Carl's house. Sorry, Carl."

I'm also experiencing a massage outage. Please send masseuse.

Goddammit some (most?) days I can't type. "Massive"

Re: Google Cloud Is Down

#500
post #185

Earlier quoted context omitted.

It's a trade-off of risks. Leaving a key under the may could lead to a security breach.

So can a compromise of a "smart" lock. It's a fake trade-off, because you're choosing between lo-tech solution and bad engineering. IoT would work better if you made the "I" part stand for "Intranet", and kept the whole thing a product instead of a service. Alas, this wouldn't support user exploitation.

Yeah, my dream device would be some standard app architecture that could run on consumer routers. You buy the router and it's your family file and print server, and also is the public portal to manage your IoT devices like cameras, locks, thermostats, and lights.
Post reply on HN