Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud Is Down

news.ycombinator.com

Post reply on HN
Print view View original

341–350 of 630 posts

Re: Google Cloud Is Down

#341

Post by luu »

actuator wrote: post #238
javabean_ wrote: post #159

Nice overview : https://downdetector.com/

Instagram is on GCloud? Facebook has a heavy investment in their own DCs, I thought they would have transitioned to them by now.

No, the graphs are highly misleading because they autoscale the y axis to the highest point and the they do very loose string matching to detect errors.

If you click through to something not on Google cloud, you see moderately elevated error rates (e.g., Instagram is up by 4x) but if you click through to something actually on Google, you see very highly elevated error rates (e.g., roughly 50000x for Snapchat).

If you read the "error reports", they actually report that Instagram isn't down (same for Twitter if you check Twitter). The error report detection seems to be just string matching. Here's an actual "error report" from downdetector that's the caused of allegedly elevated error rates:

> my twitter timeline: why isn’t snapchat working? anyone’s snapchat not working? snapchat’s being dumb. rip snapchat.

The Twitter "error report" is literally a report that Twitter isn't down.

Re: Google Cloud Is Down

#342

Post by kerng »

WC3w6pXxgGd wrote: post #107
w_s_l wrote: post #83

You know this reminds me of a bad taste that Google Sales team left when I asked for some of my billing that I was unaware of running after following a quickstart guide. AWS refunded me in the first reply on the same day! GCP sales rep just copy pasted a link to a self support survey that essentially told me, after a series of YES or NO questions that they can't refund me. So why not just tell your customers like it…

Anything created in-house at Google (GCP) is typically created by technically-proficient devs, those devs then leave the project to start something new and maintenance is left to interns and new hires. Google customer service basically doesn't care and also has no tools at their disposal to fix any issues anyway. The infinite money spout that is Google Ads has created a situation in which devs are at Google just to h…

I'm not sure why you are downvoted - seems like a reasonable insight and explanation for the drop in quality and weird decisions Google is making recently.

Re: Google Cloud Is Down

#343

Post by actuator »

sneak wrote: post #289
actuator wrote: post #238

Earlier quoted context omitted.

Instagram is on GCloud? Facebook has a heavy investment in their own DCs, I thought they would have transitioned to them by now.

Just because you run your own ram and compute doesn’t mean you run your object store, too. Running drives is a commodity now, and crypto is cheap since aes-ni. No idea if this is what instagram does or not, just in general. Drives are hot and need lots of power and it’s expensive to out-S3 S3.

Object store won't cause Instagram to go down though, no? Assets might fail to load.

Re: Google Cloud Is Down

#344

Post by fusl »

StreamBright wrote: post #317
djsumdog wrote: post #247

Earlier quoted context omitted.

Back when S3 failures would take town Reddit, parts of Twitter .. Netflix survived because they had additional availability zones. I can remember the bigger names started moving more stuff to their own data centers. AWS tries to lock people in to specific services now which makes it really difficult to migrate. It also takes a while before you get to the tipping point where hosting your own is more financially viable…

I am not sure if a single S3 outage pushed any big names into their own "datacenter". S3 has still the world record of reliability that you cannot challenge with your inhouse solutions. You can prove it otherwise. I would love to hear a solution that has the same durability, avabiality and scalability as S3. For the downvoters, please just link here the proof if you disagree. Here are the S3 numbers: https://aws.amaz…

> For the downvoters, please just link here the proof if you disagree.

https://wasabi.com/

Re: Google Cloud Is Down

#346

Post by dltmrcd »

curiouscats wrote: post #231
gowld wrote: post #185

Earlier quoted context omitted.

It's a trade-off of risks. Leaving a key under the may could lead to a security breach.

I am pretty sure there are smart locks that don't rely on an active connection to the cloud. The lock downloads keys when it has a connection and a smartphone can download keys. This means they work even if no active internet connection at the time the person tries to open. If the connection was dead the entire time between creating the new key and the person trying to use the lock it still wouldn't work. If there ar…

It may not be worth the complexity to give users the choice. If I were to issue keys to guests this way I would want my revocations to be immediately effective no matter what. Guest keys requiring a working network is a fine trade-off.

Re: Google Cloud Is Down

#347

Post by snicker7 »

StreamBright wrote: post #317
djsumdog wrote: post #247

Earlier quoted context omitted.

Back when S3 failures would take town Reddit, parts of Twitter .. Netflix survived because they had additional availability zones. I can remember the bigger names started moving more stuff to their own data centers. AWS tries to lock people in to specific services now which makes it really difficult to migrate. It also takes a while before you get to the tipping point where hosting your own is more financially viable…

I am not sure if a single S3 outage pushed any big names into their own "datacenter". S3 has still the world record of reliability that you cannot challenge with your inhouse solutions. You can prove it otherwise. I would love to hear a solution that has the same durability, avabiality and scalability as S3. For the downvoters, please just link here the proof if you disagree. Here are the S3 numbers: https://aws.amaz…

It's not so much AWS vs. in-house. But AWS (or GCP/DO/etc.) vs. multi/hybrid solutions. The latter of which would presumably have lower downtime.

Re: Google Cloud Is Down

#349

Post by _xerxes_ »

Nest is down too, not surprising given they are part of Google. What I don't understand is why I can't still control my devices over my local network. Why does the system even require access to Google servers?

Re: Google Cloud Is Down

#350

Post by ls612 »

boulos wrote: post #151
boulos wrote: post #63

Disclosure: I work on Google Cloud (but disclaimer, I'm on vacation and so not much use to you!). We're having what appears to be a serious networking outage. It's disrupting everything, including unfortunately the tooling we usually use to communicate across the company about outages. There are backup plans, of course, but I wanted to at least come here to say: you're not crazy, nothing is lost (to those concerns do…

To clarify something: this outage doesn’t appear to be global, but it is hitting us particularly hard in parts of the US. So for the folks with working VMs in Mumbai, you’re not crazy. But for everyone with sadness in us-central1, the team is on it.

I’m from the US and in Australia right now. Both me and my friends in the US are experiencing outages across google properties and Snapchat, so it’s pretty global.
Post reply on HN