Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Kubernetes Engine's third consecutive day of service disruption

status.cloud.google.com

Post reply on HN
Print view View original

121–130 of 419 posts

Re: Google Kubernetes Engine's third consecutive day of service disruption

#121

Post by Jedi72 »

usmannk wrote: post #118

We had an issue a few weeks ago where the google front-end servers were mangling responses from Pub/Sub and returning 502 responses, making the service completely unusable and knocking over a number of things we have running in production. Despite paying for enterprise support and having in a P1 ticket, we had to spend Friday to Sunday gathering evidence to prove to the support staff that there was indeed a problem,…

They work for Google so obviously they are much smarter than you. If theres a problem its probably the customers fault. /sarcasm

Re: Google Kubernetes Engine's third consecutive day of service disruption

#122

Post by origami777 »

hacknat wrote: post #52

Question to Google employees: Why do you guys suffer global outages? This is your 2nd major global outage in less than 5 years. I’m sorry to say this, but it is the equivalent of going bankrupt from a trust perspective. I need to see some blog posts about how you guys are rethinking whatever design can lead to this - twice - or you are never getting a cent of money under my control. You have the most feature rich clo…

Most feature rich cloud? I think that title belongs to AWS.

Re: Google Kubernetes Engine's third consecutive day of service disruption

#124

Post by rdtsc »

marcinzm wrote: post #36
justinsb wrote: post #29

Hi - I work at Google on GKE - sorry about the problems you're experiencing. There's a lot of people inside Google looking into this right now! It looks like the UI issue was actually fixed, and that we just didn't update the status dashboard correctly. But we're double checking that and looking into some of the additional things you all have reported here.

I appreciate all the effort you're putting in and I understand such situations can be stressful but user's having to depend on someone responding on hacker news for status updates seems really amateur for an organization the size of google.

Strangely and sadly with gmail account blocking and other such issues HN and Twitter is often better way to get Google's support than to contact support.

Re: Google Kubernetes Engine's third consecutive day of service disruption

#125

Post by DoreenMichele »

izacus wrote: post #57
marcinzm wrote: post #26

Earlier quoted context omitted.

Fair point but still seems odd that the people providing updates took the weekend off during a large scale customer impacting issue. I'm sure all the people spending the weekend trying to mitigate the impact of this on their infrastructure would love to have timely updates.

It's weekend, why wouldn't you take it off? It's just silly software.

A quick search suggests that silly software is used by dozens of companies classified as hospitals or healthcare.

Let's hope you don't have a life threatening medical emergency that can't wait near an affected healthcare facility while that silly software is down.

https://idatalabs.com/tech/products/kubernetes

Re: Google Kubernetes Engine's third consecutive day of service disruption

#128

Post by joshuamorton »

michaelt wrote: post #120
joshuamorton wrote: post #93

Earlier quoted context omitted.

Incident manager isn't public comms person. The person updating that status dashboard may or may not be an engineer, the IM certainly is.

At a smaller, less corporate company the engineer/public comms person dividing line would not be so ossified the divide couldn't be bridged when the situation called for it.

I think the broader point here is that they (whomever they is) don't think the situation calls for it. That's why they said the next update would be on Monday on Friday.

You may feel that's a bad decision, but I doubt that people are in a panic because they can't push out an update that would not be noticably different from the last one.

Just to clarify, what should this update contain?

Re: Google Kubernetes Engine's third consecutive day of service disruption

#129

Post by brian-armstrong »

justinsb wrote: post #50
Jedi72 wrote: post #21

"The data says engagement is down 46%, I think its time we drop the product." - Someone at Google right now, probably.

I can assure you that's not the case! Also, while people like to repeat this meme, Google Cloud does have a formal deprecation policy ( https://cloud.google.com/terms/ ), whose intent is to give you some assurances. (I work at Google, on GKE, though I am not a lawyer and thus don't work on the deprecation policy)

What happens when they suddenly deprecate the deprecation policy?

Re: Google Kubernetes Engine's third consecutive day of service disruption

#130

Post by joshuamorton »

rorykoehler wrote: post #119
joshuamorton wrote: post #93

Earlier quoted context omitted.

Incident manager isn't public comms person. The person updating that status dashboard may or may not be an engineer, the IM certainly is.

They should take a few mins out of their weekend to update the dashboard regardless. If my small 25 employee company can do it Google can do it.

See my other comment, to say what, exactly?

That's yes, it's still being investigated?

Post reply on HN