Google outage – resolved • Hacker News Board

Re: Google outage – resolved

#581

Post by Jumziey » Mon, Dec 14, 2020, 12:56 PM UTC

It's impressive how fast they are at fixing outages, barely a coffee break required. I'm almost starting to believe in this DevOps thing...

Re: Google outage – resolved

#582

Post by mozinator » Mon, Dec 14, 2020, 12:56 PM UTC

ecf wrote: post #540

mozinator wrote: post #322

edit: added details edit: redacted my phone number edit: big mistake to add phone number edit: I think illic is right, probably not me edit: removed details

Or the more realistic scenario is that the outage happened before you did getIamPolicy which is what caused the garbage data.

That makes sense

Re: Google outage – resolved

#583

Post by ketamine__ » Mon, Dec 14, 2020, 12:56 PM UTC

faeyanpiraat wrote: post #46

tosh wrote: post #24

Just checked https://www.google.com/appsstatus all green, which does not reflect reality for me (e.g. Gmail is down) edit: shows how incredibly difficult introspection is

HN is my goto status page when these things happen, never failed to provide up-to-date reliable information.

There are other sites that can check for you

Re: Google outage – resolved

#584

Post by exhaze » Mon, Dec 14, 2020, 12:56 PM UTC

Seems like the incident lasted from 04:00 AM PST to 04:30 AM PST, at least for BigQuery (sorry, image timestamps are in JST, since I'm in Japan): https://imgur.com/a/kiHED6v

Re: Google outage – resolved

#585

Post by crosspanda » Mon, Dec 14, 2020, 12:56 PM UTC

tosh wrote: post #24

Just checked https://www.google.com/appsstatus all green, which does not reflect reality for me (e.g. Gmail is down) edit: shows how incredibly difficult introspection is

Definitely i agree with you. Although the statuses are updated later, it may cause many problems if it is not instantaneous.

Re: Google outage – resolved

#586

Post by vultour » Mon, Dec 14, 2020, 12:57 PM UTC

rakoo wrote: post #577

rocho wrote: post #532

Earlier quoted context omitted.

That figure seems way too low, what are your sources on it?

World GDP was ~$90B last year ( https://databank.worldbank.org/data/download/GDP.pdf ), which averages to ~$150M/minute

That's trillion not billion

Re: Google outage – resolved

#587

Post by PascLeRasc » Mon, Dec 14, 2020, 12:57 PM UTC

[deleted]

Re: Google outage – resolved

#588

Post by yreg » Mon, Dec 14, 2020, 12:57 PM UTC

hooande wrote: post #571

hemmert wrote: post #73

Google Reader and Google Wave also seem to be affected.

...Google Reader still exists?

It's a joke

Re: Google outage – resolved

#589

Post by xyst » Mon, Dec 14, 2020, 12:58 PM UTC

jacquesm wrote: post #196

So, anybody still feel like arguing that 'the cloud' is a viable back-up? Or is that a sore point right now? Just for a moment imagine: what if it never comes back again? Of course it will, - at least, it better - but what if it doesn't? And if it does, are you going to take countermeasures in case it happens again or is it just going to be 'back to normal' again?

People would probably argue a “multi cloud” solution. Have your infrastructure be “cloud agnostic” and this sort of problem would be avoided.

There was actually a project called “spinnaker” that was supposed to solve this problem.

Whether the cost of paying 2 or more cloud providers is worth it for most companies is up in the air.

Re: Google outage – resolved

#590

Post by Radim » Mon, Dec 14, 2020, 12:59 PM UTC

rakoo wrote: post #564

Dayshine wrote: post #218

Earlier quoted context omitted.

I feel like HN's server aren't very happy with everybody doing this though. Stop bullying the servers :(

This wouldn't be a problem if HN migrated to a blockchain written in go for performance

Would never work without a powerful AI engine in the middle.