Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GCP Outage

status.cloud.google.com

Post reply on HN
Print view View original

521–530 of 539 posts

Re: GCP Outage

#521

Post by __turbobrew__ »

IX-103 wrote: post #483
a2128 wrote: post #468

Earlier quoted context omitted.

Some day Cloudflare will depend on GCP and GCP will depend on Cloudflare and AWS will rely on one of the two being online and Cloudflare will also depend on AWS and the internet will go down and no one will know how to restart it

Supposedly something like this already happened inside Google. There's a distributed data store for small configs read frequently. There's another for larger configs that are rarely read. The small data store depends on a service that depends on the large data store. The large data store depends on the small data store. Supposedly there are plans for how to conduct a "cold" start of the system, but as far as I know i…

The trick there is you take the relevant configs and serialize them to disk periodically, and then in a bootstrap scenario you use the configs on disk.

Presumably for the infrequently read configs you could do this so the service with frequently read configs can bootstrap without the service for infrequently read configs.

Re: GCP Outage

#522

Post by christianqchung »

cryptonector wrote: post #353
atonse wrote: post #26

Getting a lot of errors for Claude Sonnet 4 (Cursor) and Gemini Pro. Nooooo I'm going to have to use my brain again and write 100% of my code like a caveman from December 2024.

Devs before June 12, 2025: "Ai? Pfft, hallucination central. They'll never replace me!" Devs during June 12, 2025 GCP outage: "What, no AI?! Do you think I'm a slave?!"

Goomba fallacy

Re: GCP Outage

#523

Post by [deleted] »

augbog wrote: post #382

Cloudflare Outage also just updated > Cloudflare’s critical Workers KV service went offline due to an outage of a 3rd party service that is a key dependency. As a result, certain Cloudflare products that rely on KV service to store and disseminate information

[deleted]

Re: GCP Outage

#524

Post by boneitis »

peanut-walrus wrote: post #464
johanyc wrote: post #458

Earlier quoted context omitted.

Why

Downdetector does not actually monitor the services. It aggregates user reports from socials etc. For large-scale incidents, the reports get really noisy and it will show that basically everything is down.

I thought that was the whole premise of Downdetector, no? User reports, because first-party status updates are tightly controlled by those first parties?

Was not basically everything (hyperbolically speaking, of course) practically impacted today?

How much weight really comes from those social media posts? Is there an indirect effect of people reading these posts, then flocking to hit the report button, sight unseen?

Re: GCP Outage

#525

Post by refulgentis »

9rx wrote: post #491
tfsh wrote: post #139

Earlier quoted context omitted.

It's not. You might be joking, but that comment still isn't helpful. My understanding is this is part of Google's internal PSD offering (Public Status Board) which uses SCS (Static Content Service) behind GFE (Google Frontend) which is hosted on Borg, and deploys other large scale apps such as Search, Drive, YouTube, etc.

How could it not be helpful given that it gave you reason to provide more details that you wouldn't have otherwise shared? You may not have thought this through. There is nothing more helpful. Unless you think your own comment isn't helpful, but then...

Because "It's good to lie because it makes people correct me" is a joke about IRC, not a viable stable game-theoretic optimal position.

Re: GCP Outage

#526

Post by syllogism »

__turbobrew__ wrote: post #521
IX-103 wrote: post #483

Earlier quoted context omitted.

Supposedly something like this already happened inside Google. There's a distributed data store for small configs read frequently. There's another for larger configs that are rarely read. The small data store depends on a service that depends on the large data store. The large data store depends on the small data store. Supposedly there are plans for how to conduct a "cold" start of the system, but as far as I know i…

The trick there is you take the relevant configs and serialize them to disk periodically, and then in a bootstrap scenario you use the configs on disk. Presumably for the infrequently read configs you could do this so the service with frequently read configs can bootstrap without the service for infrequently read configs.

Like a backup generator for inputs. Makes sense.

Re: GCP Outage

#527

Post by rldjbpin »

cddotdotslash wrote: post #506
pancomplex wrote: post #336

thank god hn is hosted on a single bare metal server, free of all this bloat.

I can recall plenty of times HN has been down.

it has already been facing some issues in the past couple weeks. e.g. the "front" page was broken randomly for past dates (like https://news.ycombinator.com/front?day=2025-05-30 or https://news.ycombinator.com/front?p=2)

as per their api documentation [1], it might be linked to firebase, which might explain this?

[1] https://github.com/HackerNews/API

Re: GCP Outage

#528

Post by detaro »

rldjbpin wrote: post #527
cddotdotslash wrote: post #506

Earlier quoted context omitted.

I can recall plenty of times HN has been down.

it has already been facing some issues in the past couple weeks. e.g. the "front" page was broken randomly for past dates (like https://news.ycombinator.com/front?day=2025-05-30 or https://news.ycombinator.com/front?p=2 ) as per their api documentation [1], it might be linked to firebase, which might explain this? [1] https://github.com/HackerNews/API

the API has afaik nothing to do with HN itself, its a read-only copy for API users, not what backs the actual site.

Re: GCP Outage

#529

Post by solardev »

IX-103 wrote: post #483
a2128 wrote: post #468

Earlier quoted context omitted.

Some day Cloudflare will depend on GCP and GCP will depend on Cloudflare and AWS will rely on one of the two being online and Cloudflare will also depend on AWS and the internet will go down and no one will know how to restart it

Supposedly something like this already happened inside Google. There's a distributed data store for small configs read frequently. There's another for larger configs that are rarely read. The small data store depends on a service that depends on the large data store. The large data store depends on the small data store. Supposedly there are plans for how to conduct a "cold" start of the system, but as far as I know i…

Just put them in Workers KV... oh wait

Re: GCP Outage

#530

Post by 9rx »

refulgentis wrote: post #525
9rx wrote: post #491

Earlier quoted context omitted.

How could it not be helpful given that it gave you reason to provide more details that you wouldn't have otherwise shared? You may not have thought this through. There is nothing more helpful. Unless you think your own comment isn't helpful, but then...

Because "It's good to lie because it makes people correct me" is a joke about IRC, not a viable stable game-theoretic optimal position.

Cunningham's Law emerged in the newsgroups era, well predating the existence of IRC.

Of course, I recognize that you purposefully pulled the Cunningham's Law trigger so that you, too, would gain additional knowledge that nobody would have told you about otherwise, as one logically would. And that you played it off as some kind of derision towards doing that all while doing it yourself made it especially funny. Well done!

Post reply on HN