Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gmail and Google Drive Outage

google.com

Post reply on HN
Print view View original

61–70 of 88 posts

Re: Gmail and Google Drive Outage

#61

Post by jefftk »

londons_explore wrote: post #48
jordanthoms wrote: post #30

There was an outage August 19th, 2019 - almost 1 year ago to the day. As I posted at the time: "Google often has a outage or two around this time of the year when all the US schools come back and millions of students log in at the same time." My pet theory wasn't too popular but I'm going to stick with it :) 1- https://news.ycombinator.com/item?id=20740997

Google does a once per year disaster recovery training... They do things like deliberately turn off datacenters with no warning. Sometimes failover systems don't work as intended. Was that this week?

It was not this week, sorry!

Re: Gmail and Google Drive Outage

#62

Post by brown9-2 »

piahoo wrote: post #56
cezart wrote: post #24

The status from Google Cloud status page offers a bit more technical details of what happened: "We are experiencing an issue with Google Kubernetes Engine (GKE) clusters using node auto-provisioning becoming stuck during node version upgrades. Node auto-upgrades have been disabled temporarily." https://status.cloud.google.com/

Does it mean, that Gmail is hosted on GKE?

No, it’s likely the GKE incident is caused by a dependency that Gmail also has.

Re: Gmail and Google Drive Outage

#64

Post by 1f60c »

londons_explore wrote: post #57
jordanthoms wrote: post #54

Earlier quoted context omitted.

Ok, I definitely want to know how you discovered that... (and found one of those 500 people to verify?)

Not sure if this is still the case, but if you did this a couple of times, your account data would be permanently migrated to an instance with more CPU and RAM allocated - you'd also be in with all the other badly behaved accounts, so reliability goes down lots. The benefit was much quicker complex searches, and being able to bulk label or delete emails without it taking minutes or hours. Don't believe me how slow it…

I find it hard to believe that Gmail will always serve certain users from the same machines, especially in this day and age, with “cattle, not pets” and ephemeral containers.

I’m sure they have machines that are only used to serve G Suite and Google One customers, and maybe some other VIPs, but regular heavy users? It sounds like an urban legend to me.

Re: Gmail and Google Drive Outage

#65

Post by iagooar »

RachelF wrote: post #55
naedish wrote: post #22

I'm guessing this outage will allow GSuite customers to claim Service Credits under the SLA - does anyone have any experience with doing so? Google's documentation is lacking in detail[0]. [0] https://gsuite.google.com/intl/en/terms/sla.html

Good luck with that - read GSuite's terms, they, Google, define what an outage is, not the customer.

It depends on the specific country's laws - in Germany you can not write anything you want into a contract and call it valid. In this case, you could still sue Google if according to industry standard definition it actually was an outage.

Re: Gmail and Google Drive Outage

#66

Post by 9nGQluzmnq3M »

cezart wrote: post #24

The status from Google Cloud status page offers a bit more technical details of what happened: "We are experiencing an issue with Google Kubernetes Engine (GKE) clusters using node auto-provisioning becoming stuck during node version upgrades. Node auto-upgrades have been disabled temporarily." https://status.cloud.google.com/

That issue is unrelated. This is the correct one: https://status.cloud.google.com/incident/zall/20008

Re: Gmail and Google Drive Outage

#68

Post by grumple »

iagooar wrote: post #65
RachelF wrote: post #55

Earlier quoted context omitted.

Good luck with that - read GSuite's terms, they, Google, define what an outage is, not the customer.

It depends on the specific country's laws - in Germany you can not write anything you want into a contract and call it valid. In this case, you could still sue Google if according to industry standard definition it actually was an outage.

The same is true in the US but some people like to pretend contracts mean everything here.

Re: Gmail and Google Drive Outage

#69

Post by organsnyder »

1f60c wrote: post #64
londons_explore wrote: post #57

Earlier quoted context omitted.

Not sure if this is still the case, but if you did this a couple of times, your account data would be permanently migrated to an instance with more CPU and RAM allocated - you'd also be in with all the other badly behaved accounts, so reliability goes down lots. The benefit was much quicker complex searches, and being able to bulk label or delete emails without it taking minutes or hours. Don't believe me how slow it…

I find it hard to believe that Gmail will always serve certain users from the same machines, especially in this day and age, with “cattle, not pets” and ephemeral containers. I’m sure they have machines that are only used to serve G Suite and Google One customers, and maybe some other VIPs, but regular heavy users? It sounds like an urban legend to me.

The data has to be sharded somehow, though. You might not be hitting the same exact machine, especially for the frontend, but your data isn't just magically everywhere in "the cloud".

Re: Gmail and Google Drive Outage

#70

Post by jacobr1 »

1f60c wrote: post #64
londons_explore wrote: post #57

Earlier quoted context omitted.

Not sure if this is still the case, but if you did this a couple of times, your account data would be permanently migrated to an instance with more CPU and RAM allocated - you'd also be in with all the other badly behaved accounts, so reliability goes down lots. The benefit was much quicker complex searches, and being able to bulk label or delete emails without it taking minutes or hours. Don't believe me how slow it…

I find it hard to believe that Gmail will always serve certain users from the same machines, especially in this day and age, with “cattle, not pets” and ephemeral containers. I’m sure they have machines that are only used to serve G Suite and Google One customers, and maybe some other VIPs, but regular heavy users? It sounds like an urban legend to me.

It isn't about the compute. This is about the databases. I don't see it as stretch that google is using a sharding strategy for their datastores.
Post reply on HN