Live data from Hacker News

Cloudflare outage on December 5, 2025

blog.cloudflare.com

351–360 of 601 posts

Re: Cloudflare outage on December 5, 2025

#351
post #325

Earlier quoted context omitted.

Management doesn't like when things like this are automated. They want to "manage" the outage/production/etc numbers before letting them out.

100% — will never be automated :)

Still room for someone to claim the niche of the Porsche horsepower method in outage reporting - underpromise, overdeliver.

Re: Cloudflare outage on December 5, 2025

#352
post #125

Suggestion for Cloudflare: Create an early adopter option for free accounts. Benefit: Earliest uptake of new features and security patches. Drawback: Higher risk of outages. I think this should be possible since they already differentiate between free, pro and enterprise accounts. I do not know how the routing for that works but I bet they could do this. Think crowd-sourced beta testers. Also a perk for anything PCI…

They do in some way because the LaLiga blocking problems in Spain don’t affect the paid accounts=large websites.

An other suggestion is to do it along night shift in every country, right now they only take into account EEUU night.

Re: Cloudflare outage on December 5, 2025

#353
post #327

Earlier quoted context omitted.

> They really need to figure out a way to correlate global configuration changes to the errors they trigger as fast as possible. This is what jumped out at me as the biggest problem. A wild west deployment process is a valid (but questionable) business decision, but if you do that then you need smart people in place to troubleshoot and make quick rollback decisions. Their timeline: > 08:47: Configuration change deplo…

> 2 minutes for their automated alerts to fire is terrible I take exception to that, to be honest. It's not desirable or ideal, but calling it "terrible" is a bit ... well, sorry to use the word ... entitled. For context, I have experience running a betting exchange. A system where it's common for a notable fraction of transactions in a medium-volume event to take place within a window of less than 30 seconds. Vast m…

Prometheus compatible but more reliable? Sell it to me!

Re: Cloudflare outage on December 5, 2025

#354
post #347

Earlier quoted context omitted.

Artifical limits because they are on the free plan?

Artifical limits, because they have 40 paid licenses that they can not use, because of a non-disclosed assignment limit that is NOT mentioned in the pricing page nor in the ToS. A lot of people are angry about this, and I think it's borderline illegal: https://devforum.zoom.us/t/you-have-exceeded-the-limit-of-li... You pay for something, and you can't use it.

This is why we never changed the licenses, we just made long-running identical ID meetings that everyone can join.

But we’re moving away as it’s only going to get worse.

Re: Cloudflare outage on December 5, 2025

#355

Earlier quoted context omitted.

What "hundreds of billions of dollars"? Cloudflare's annual revenue is around $2 billion, and they are not yet profitable.

That was admittedly hyperbole, but since we're talking about a company with assets and revenue in the billions I'm not sure it matters. The fact remains that a lack of money/resources is not their problem.

They don't have unlimited resources. They have ~5000 employees. That's not small but it's not huge either. For sake of comparison, Google hit that headcount level literally 20 years ago.

Re: Cloudflare outage on December 5, 2025

#356
Classic. Things always get worse before they get better. I remember when Netflix was going through their annus horribilis, and AWS before that, and Twitter before that, and so on. Everyone goes through this. Good luck to you guys getting to FL2 quickly enough that this class of error reduces.

Re: Cloudflare outage on December 5, 2025

#357

Earlier quoted context omitted.

This is the case with just about every status page I’ve ever seen. It takes them a while to realize there’s really a problem and then to update the page. One day these things will be automated, but until then, I wouldn’t expect more of Cloudflare than any other provider. What’s more concerning to me is that now we’ve had AWS, Azure, and CloudFlare (and CliudFlare twice) go down recently. My gut says: 1. developers an…

Management doesn't like when things like this are automated. They want to "manage" the outage/production/etc numbers before letting them out.

There's no sweet spot I've found. I don't work for Cloudflare but when I did have a status indicator to maintain, you could never please everyone. Users would complain when our system was up but a dependent system was down, saying that our status indicator was a lie. "Fixing" that by marking our system as down or degraded whenever a dependent system was down led to the status indicator being not green regularly, causing us to unfairly develop a reputation as unreliable (most broken dependencies had limited blast radius). The juice no longer seemed worth the squeeze and we gave up on automated status indicators.

Re: Cloudflare outage on December 5, 2025

#358
post #347

Earlier quoted context omitted.

Artifical limits, because they have 40 paid licenses that they can not use, because of a non-disclosed assignment limit that is NOT mentioned in the pricing page nor in the ToS. A lot of people are angry about this, and I think it's borderline illegal: https://devforum.zoom.us/t/you-have-exceeded-the-limit-of-li... You pay for something, and you can't use it.

This is why we never changed the licenses, we just made long-running identical ID meetings that everyone can join. But we’re moving away as it’s only going to get worse.

That's a cool work-around.

What I don't like, is that whenever you contact Zoom, their representatives are taught to say one thing: buy more licenses.

Not only that, but their API/pricing is specifically designed to cover edge-cases that will force you to buy a license.

For example, they don't expose an API to assign a co-host. You can do that via the UI, manually, but not via the API.

Can you share which solution are you moving to?

Re: Cloudflare outage on December 5, 2025

#359

Earlier quoted context omitted.

You call it extortion of the AI companies, but isn’t stealing/crawling/hammering a site to scrape their content to resell just as nefarious? I would say Cloudflare is giving these site owners an option to protect their content and as a byproduct, reduce their own costs of subsidizing their thieves. They can choose to turn off the crawl protection. If they aren't, that tells you that they want it, doesn’t it?

>You call it extortion of the AI companies, but isn’t stealing/crawling/hammering a site to scrape their content to resell just as nefarious? You can easily block ChatGPT and most other AI scrapers if you want: https://habeasdata.neocities.org/ai-bots

No you cannot! I blocked all of the user agents on a community wiki I run, and the traffic came back hours later masquerading as Firefox and Chrome. They just fucking lie to you and continue vacuuming your CPU.

Re: Cloudflare outage on December 5, 2025

#360

I noticed this outage last night (Cloudflare 500s on a few unrelated websites). As usual, when I went to Cloudflare's status page, nothing about the outage was present; the only thing there was a notice about the pre-planned maintenance work they were doing for the security issue, reporting that everything was being routed around it successfully.

This is the case with just about every status page I’ve ever seen. It takes them a while to realize there’s really a problem and then to update the page. One day these things will be automated, but until then, I wouldn’t expect more of Cloudflare than any other provider. What’s more concerning to me is that now we’ve had AWS, Azure, and CloudFlare (and CliudFlare twice) go down recently. My gut says: 1. developers an…

>It takes them a while to realize there’s really a problem and then to update the page.

Not really, they're just lying. I mean yes of course they aren't oracles who discover complex problems in instant of the first failure, but naw they know when well there are problems and significantly underreport them to the extent they are are less "smoke alarms" and more "your house has burned down and the ashes are still smoldering" alarms. Incidents are intentionally underreported. It's bad enough that there ought to be legislation and civil penalties for the large providers who fail to report known issues promptly.

Post reply on HN