Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Down Again – and DownDetector Is Also Down

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 177 posts

Re: Cloudflare Down Again – and DownDetector Is Also Down

#141

Post by pessimizer »

bflesch wrote: post #71

How are these clowns deploying stuff on a Friday, it is unbelievable to me. It is not even funny any more. It seems cloudflare is held together by marketing only. They should stop all of these stupid initiatives and keep their stack simple. And I'm 100% sure the management responsible for this is already fueling up the ferraris to drive to their beach house. All of us make them rich and they keep on enshittifying the…

If you are a monopoly, there is no incentive to do anything well. You've saturated the market, the incentive is to cut costs.

In fact, there are incentives for public failures: they'll help the politicians that you bought sell the legislation that you wrote explaining how national security requires that the taxpayer write a check to your stockholders/owners in return for nothing.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#142

Post by kylecazar »

gosub100 wrote: post #135
cowsandmilk wrote: post #118

Earlier quoted context omitted.

I’m sick of people saying this. The truth at every cloud provider: 1. There were outages under the old guard. 2. The new guard is operating systems that are larger than what the old guard operated.

You don't think companies try to save costs by hiring the cheapest and dumbest people?

I don't think that's the exact mechanism, no.

They might go on a hiring freeze, cancel a role, or in some cases pass on someone asking too much... But I don't think any major players are actively out trawling for "cheap and dumb". Certainly not Cloudflare, AWS and Google.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#143

Post by amiga386 »

winternewt wrote: post #88
noosphr wrote: post #75

I know I shouldn't, but I really can't help myself: https://blog.cloudflare.com/20-percent-internet-upgrade/

What are you implying by linking to that article?

In the chain of events that led to Cloudflare's largest ever outage, code they'd rewritten from C to Rust was significant factor. There are, of course, other factors that meant the Rust-based problem was not mitigated.

They expected a maximum config size but an upstream error meant it was much larger than normal. Their Rust code parsed a fraction of the config, then did ".unwrap()" and panicked, crashing the entire program.

This validated a number of things that programmers say in response to Rust advocates who relentlessly badger people in pursuit of mindshare and adoption:

* memory errors are not the only category of errors, or security flaws. A language claiming magic bullets for one thing might be nonetheless be worse at another thing.

* there is no guarantee that if you write in your code will have fewer errors. If anything, you'll add new errors during the rewrite

* Rust has footguns like any other language. If it gains common adoption, there will be doofus programmers using it too, just like the other languages. What will the errors of Rust doofuses look like, compared to C, C++, C#, Java, JavaScript, Python, Ruby, etc. doofuses?

* availability is orthagonal to security. While there is a huge interest in remaining secure, if you design for "and it remains secure because it stops as soon as there's an error", have you considered what negative effects a widespread outage would cause?

Re: Cloudflare Down Again – and DownDetector Is Also Down

#144

Post by gosub100 »

renegade-otter wrote: post #83
_fizz_buzz_ wrote: post #78

Earlier quoted context omitted.

Just because someone is on an H1B visa doesn't mean they know less. It's a bit rich to blame this on foreign workers even though nothing is known about who or what caused this outage.

The problem with H1B is that these people are effectively prisoners. The market is not so hot right now even for those who have leverage, but combine it with the visa system and you get this "gotta do the needful" attitude to please the bosses, rushing broken fixes to production.

I see this directly on my team. The h1bs get bullied by their boss (it's a split team, I work with him but don't report to him) and they don't say anything because he could effectively have them deported. At least 2 of them have kids here and perhaps the others do. So not only does it incentivize the bully to do it, but it traps them to just take it for their family. I openly talk shit back to him because he can't deport me.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#145

Post by stravant »

jsheard wrote: post #120
smallerize wrote: post #90

Earlier quoted context omitted.

And just live with high latency?

The website you're using right now is hosted from a single location without any kind of CDN in front, so unless by coincidence you happen to live next door then you seem to be managing. CDNs do help, but just not bundling 40MB of Javascript or doing 50 roundtrips to load a page can go a long way.

The website you're using right now is possibly the single least want of a CDN out of any of the sites I regularly visit.

What other popular site has zero images or video to speak of?

Re: Cloudflare Down Again – and DownDetector Is Also Down

#146

Post by numlock86 »

bo1024 wrote: post #126
peanut-walrus wrote: post #79

Earlier quoted context omitted.

Cloudflare is widely used because it's the easiest way to run a website for free or expose local services to internet. I think for most cloudflare users, the ddos protection is not the main reason they're using it.

Cloudflare hosts websites for free?

Yes, they have free plans.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#147

Post by YouAreWRONGtoo »

huijzer wrote: post #56
yatralalala wrote: post #48

Internet is no longer decenstralised. Some interesting DNS data https://news.ycombinator.com/item?id=46159249

Hence why I wrote a post on 18th of Nov (previous Cloudflare outage): https://huijzer.xyz/posts/123/do-not-put-your-site-behind-cl... . That blog post made it to the front page of HN and my site did not go down. Nor did any DDoS network take the site out even though I also challenged them last time by commenting that I would be okay with a DDoS. I would figure out a way around it. In general, marketing often works vi…

[dead]

Re: Cloudflare Down Again – and DownDetector Is Also Down

#148

Post by ineedasername »

klas_segeljakt wrote: post #24
JoeWiltshire wrote: post #11

https://downdetectorsdowndetectorsdowndetectorsdowndetector.... reports that https://downdetectorsdowndetectorsdowndetector.com/ is down, guessing downdetectorsdowndetectorsdowndetector runs via cloudflare!

It's downdetectors all the way down

Hot take: this is a byproduct of monolithic detector ecosystems. We need diversification of detectors. I propose: sidewaysdetector.com

Re: Cloudflare Down Again – and DownDetector Is Also Down

#149

Post by antoinealb »

jsheard wrote: post #107
noosphr wrote: post #75

I know I shouldn't, but I really can't help myself: https://blog.cloudflare.com/20-percent-internet-upgrade/

It could always be worse: https://en.wikipedia.org/wiki/Cloudbleed They haven't had an incident that bad since they switched from C to Rust.

The linked article is precisely about how in 2024 they started rewriting their proxy layer from nginx (a C app). While "They haven't had an incident that bad since they switched from C to Rust." might be true, it has also been almost 9 years since cloudbleed, of which 8 were in C world.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#150

Post by seydor »

JoeWiltshire wrote: post #11

https://downdetectorsdowndetectorsdowndetectorsdowndetector.... reports that https://downdetectorsdowndetectorsdowndetector.com/ is down, guessing downdetectorsdowndetectorsdowndetector runs via cloudflare!

Right now the order is reversed.

Look, i think we need a resilient system that routes packets via multiple possible pathways, preferably all of them, so that ideally nothing is ever fully down. We can name that system the undownnet

Post reply on HN