Live data from Hacker News

Cloudflare outage on December 5, 2025

blog.cloudflare.com

391–400 of 601 posts

Re: Cloudflare outage on December 5, 2025

#391
post #367

Earlier quoted context omitted.

This feels like such a valid solution and is how past $dayjobs released things: send to the free users, rollout to Paying Users once that's proven to not blow up.

If your target is availability, that's correct. If your target is security, then _assuming your patch is actually valid_ you're giving better security coverage for free customers than to your paying ones. Cloudflare is both, and their tradeoffs seem to be set on maximizing security at cost of availability. And it makes sense. A fully unavailable system is perfectly secure.

[deleted]

Re: Cloudflare outage on December 5, 2025

#392

This is architectural problem, the LUA bug, the longer global outage last week, a long list of earlier such outages only uncover the problem with architecture underneath. The original, distributed, decentralized web architecture with heterogeneous endpoints managed by myriad of organisations is much more resistant to this kind of global outages. Homogeneous systems like Cloudflare will continue to cause global outage…

Bro, but how do we make shareholder value if we don't monopolize and enshittify everything

Re: Cloudflare outage on December 5, 2025

#393

Earlier quoted context omitted.

You call it extortion of the AI companies, but isn’t stealing/crawling/hammering a site to scrape their content to resell just as nefarious? I would say Cloudflare is giving these site owners an option to protect their content and as a byproduct, reduce their own costs of subsidizing their thieves. They can choose to turn off the crawl protection. If they aren't, that tells you that they want it, doesn’t it?

>You call it extortion of the AI companies, but isn’t stealing/crawling/hammering a site to scrape their content to resell just as nefarious? You can easily block ChatGPT and most other AI scrapers if you want: https://habeasdata.neocities.org/ai-bots

How are you this naive? Do you really think scrapers give a damn about your robots.txt?

Re: Cloudflare outage on December 5, 2025

#394

Earlier quoted context omitted.

[flagged]

Indeed, but it is what it is. Cloudflare comes out of my budget, and even with downtime, its better than not paying them. Do I want to deal with what Cloudflare offers? I do not, I have higher value work to focus on. I want to pay someone else to deal with this, and just like when cloud providers are down, it'll be back up eventually. Grab a coffee or beer and hang; we aren't savings lives, we're just building websit…

[deleted]

Re: Cloudflare outage on December 5, 2025

#395

The interesting part: After rolling out a bad ruleset update, they tried a killswitch (rolled out immediately to 100%) which was a code path never executed before: > However, we have never before applied a killswitch to a rule with an action of “execute”. When the killswitch was applied, the code correctly skipped the evaluation of the execute action, and didn’t evaluate the sub-ruleset pointed to by it. However, an…

> have never before applied a killswitch to a rule with an action of “execute” One might think a company on the scale of Cloudflare would have a suite of comprehensive tests to cover various scenarios.

[deleted]

Re: Cloudflare outage on December 5, 2025

#396

This is architectural problem, the LUA bug, the longer global outage last week, a long list of earlier such outages only uncover the problem with architecture underneath. The original, distributed, decentralized web architecture with heterogeneous endpoints managed by myriad of organisations is much more resistant to this kind of global outages. Homogeneous systems like Cloudflare will continue to cause global outage…

I’m not sure I share this sentiment.

First, let’s set aside the separate question of whether monopolies are bad. They are not good but that’s not the issue here.

As to architecture:

Cloudflare has had some outages recently. However, what’s their uptime over the longer term? If an individual site took on the infra challenges themselves, would they achieve better? I don’t think so.

But there’s a more interesting argument in favour of the status quo.

Assuming cloudflare’s uptime is above average, outages affecting everything at once is actually better for the average internet user.

It might not be intuitive but think about it.

How many Internet services does someone depend on to accomplish something such as their work over a given hour? Maybe 10 directly, and another 100 indirectly? (Make up your own answer, but it’s probably quite a few).

If everything goes offline for one hour per year at the same time, then a person is blocked and unproductive for an hour per year.

On the other hand, if each service experiences the same hour per year of downtime but at different times, then the person is likely to be blocked for closer to 100 hours per year.

It’s not really bad end user experience that every service uses cloudflare. It’s more-so a question of why is cloudflare’s stability seeming to go downhill?

And that’s a fair question. Because if their reliability is below average, then the value prop evaporates.

Re: Cloudflare outage on December 5, 2025

#397

Earlier quoted context omitted.

It's a debt though. Because people will forget it's there and then at some point someone changes a counter from milliseconds to microseconds and then the issue happens 1000 times sooner. It's never right to leave structural issues even if "they don't happen under normal conditions".

I don't think this argument makes sense. You wouldn't provision a 100GB server for a service where 1GB would do just in case unexpected conditions come up. If the requirements change, then the setup can change, doing it just because is wasteful. What if we forget is not a valid argument to over engineer and over provision.

If a fix is relatively low cost and improves the software in a way that makes it easier to modify in the future, it makes it easier to change the requirements. In aggregate these pay off.

Re: Cloudflare outage on December 5, 2025

#398

Earlier quoted context omitted.

They don't have unlimited resources. They have ~5000 employees. That's not small but it's not huge either. For sake of comparison, Google hit that headcount level literally 20 years ago.

They have enough money to buy anything they need. The CEO alone has billions. He could pay for as many employees as he wants out of his own pocket and not notice. In fact he's good at buying people, even senators.

That doesn't make sense. It would be like saying Twitter, SpaceX, and Tesla all should be incapable of engineering mistakes because their owner is rich. The world doesn't work that way.

Re: Cloudflare outage on December 5, 2025

#399

Earlier quoted context omitted.

Management doesn't like when things like this are automated. They want to "manage" the outage/production/etc numbers before letting them out.

There's no sweet spot I've found. I don't work for Cloudflare but when I did have a status indicator to maintain, you could never please everyone. Users would complain when our system was up but a dependent system was down, saying that our status indicator was a lie. "Fixing" that by marking our system as down or degraded whenever a dependent system was down led to the status indicator being not green regularly, caus…

The headline status doesn't have to be "worst of all systems". Pick a key indicator, and as long as it doesn't look like it's all green regardless of whether you're up or down, users will imagine that "green headline, red subsystems" means whatever they're observing, even if that makes the status display utterly uninterpretable from an outside perspective.
Post reply on HN