Interesting how I landed here. I was having trouble with Nest. Then I went to Down Detector. I noticed many sites having a simultaneous uptick. Then I came to HN, and found this link at the top of the front page.
I usually just go here first.
GCP Outage
161–170 of 539 posts
Re: GCP Outage
#162Sorry, after decades of being hard wired, I just installed a PCIe Wifi6 card on my desktop. Internet took a dive the second I got it connected. Must have done something wrong.
Re: GCP Outage
#163Does anyone know if instance-to-instance networking has been affected? My Redis instance has been throwing a lot of connection errors.
We're not seeing any connectivity issues between pods and vms in our vpc, but your mileage may vary.
Re: GCP Outage
#164Earlier quoted context omitted.
So even then, it should have been able to correctly report the status, it somehow shows that the status page is not automated and any change there needs to go through someone manual.
A program that updates the status page failing does not imply that the status page is manually edited. It is not like you would generate a status page on every request.
How do we know that the program is failing ?
How hard is it for the frontend to detect if the last update to the status page was made a while ago and that itself implies there is an error and should be reported ?
Re: GCP Outage
#165Smells like BGP since there are services people claim have nothing to do with GCP being affected. OpenRouter is down, Lovable is down, etc.
perhaps Lovable uses GCP somewhere in their stack?
Re: GCP Outage
#166Re: GCP Outage
#167The status page is green, but there are outages reported: https://downdetector.com/status/google-cloud/
Why can't companies be honest with being down. It helps us all out so we don't spend an hour internalizing. We are truly in gods hands. $ prod Fetching cluster endpoint and auth data. ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=503, message=Visibility check was unavailable. Please retry the request and contact support if the problem persists
Nobody gets a promotion, that's why.
Re: GCP Outage
#168The status page is green, but there are outages reported: https://downdetector.com/status/google-cloud/
[dead]
So’s Azure? https://downdetector.com/status/windows-azure/
This is where we get to learn about the one common system all of our “distributed cloud” systems rely on, isn’t it?
Re: GCP Outage
#169It's completely nuts that Firebase has this: https://status.firebase.google.com/incidents/ZcF1YDUvpdixZ2e... "Firebase Data Connect unavailable due to a known Google Cloud global outage" While the Google Cloud status page https://status.cloud.google.com/ says "No major incidents" and everything is green. So Google Cloud know there is an outage but just deem it not major enough to show it. Edit to add: within 10 minut…
Something must be preventing them updating the status page at this point. Of course they could still deem it not enough, but just from my limited tests, docker, buf, etc (it may not be GCP that is down, but it is quite the coincidence). are outright down. I'd wager that this is much more widespread.
Re: GCP Outage
#170It looks like that it is a central service @ Google called Chemist that is down. "Chemist checks the project status, activation status, abuse status, billing status, service status, location restrictions, VPC Service Controls, SuperQuota, and other policies." -> This would totally explain the error messages "visibility check (of the API) failed" and "cannot load policy" and the wide amount of services affected. cf. h…
There are multiple internet services down, not just GCP. It's just possible that this "Chemist" service is especially externally affected which is why the failures are propagating to the their internal GCP network services.