Live data from Hacker News

Cloudflare Down Again – and DownDetector Is Also Down

news.ycombinator.com

131–140 of 177 posts

Re: Cloudflare Down Again – and DownDetector Is Also Down

#133

I host my companys website on Cloudflare pages using Cloudflare's DNS. I don't want to move to 100% self hosting but I would like to have self hosted backup. Has anyone solved this?

Having a self-hosted “backup” that is ready to go at any time means having a self-hosted server that’s always on, basically. There are lots of cheap colo or VM options out there. But the problem is going to be dealing with an outage… how do you switch DNS over when your DNS provider is down?

Well, one way is to use a different DNS provider than either of your hosting options.

You can see this is getting complicated. Might be better to take the downtime.

But if I had to make a real recommendation I’m not aware of any time in the last decade that a static site deployed on AWS S3/Cloudfront would have actually been unavailable.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#134
post #71

How are these clowns deploying stuff on a Friday, it is unbelievable to me. It is not even funny any more. It seems cloudflare is held together by marketing only. They should stop all of these stupid initiatives and keep their stack simple. And I'm 100% sure the management responsible for this is already fueling up the ferraris to drive to their beach house. All of us make them rich and they keep on enshittifying the…

> How are these clowns deploying stuff on a Friday, it is unbelievable to me

I have stopped fighting this battle at work (not Cloudflare). Despite Friday being one of the most important days of the week for our customers, people still deploy the latest commit 10 minutes before they leave in the afternoon. Going on a weekend trip home to your family? No problem, just deploy and be offline for hours while you are traveling...

The response was that my way of thinking is "old school". Modern development is "fail fast" and that CI/CD with good tests and rollback fixes everything. Being afraid of deploys is "so last decade"... The problem is that our tests don't cover everything, it may not fail fast, and not all deploys can be rolled back quickly and the person who knows what their huge commit that touches multiple files actually does is unavailable!

We have had multiple issues with late afternoon deploys, but somehow we keep doing this. Funnily enough, I have noticed a pattern. Most experienced devs stops doing this after causing a couple of major downtimes, due to the massive backlash from customers while they are scrambling to fix the bug. So gradually they learn to deploy at less busy times and monitor the logs to be able to catch potential bugs early.

The problem is that not enough has learned this lesson because they have been lucky that their bugs have not been critical, or because they are too invested in their point of view to change. It seems that some individuals learn the hard way, but the organization has not learned or is reluctant to push for a change due to office politics. I decided to keep my head low and let things play out, as I simply no longer care as long as the management don't care either.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#135

The old guard has left as they we too much of an expense in this cost-cutting age... without mentors, crap creeps in and now we are seeing what happens when people don't know how things work, are in charge...

I’m sick of people saying this. The truth at every cloud provider: 1. There were outages under the old guard. 2. The new guard is operating systems that are larger than what the old guard operated.

You don't think companies try to save costs by hiring the cheapest and dumbest people?

Re: Cloudflare Down Again – and DownDetector Is Also Down

#137
post #71

How are these clowns deploying stuff on a Friday, it is unbelievable to me. It is not even funny any more. It seems cloudflare is held together by marketing only. They should stop all of these stupid initiatives and keep their stack simple. And I'm 100% sure the management responsible for this is already fueling up the ferraris to drive to their beach house. All of us make them rich and they keep on enshittifying the…

If the deployment was related to the React Server issue then maybe it was unavoidable.

Cloudflare's entire WAF depending on React is an issue in itself IMO.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#139
post #71

How are these clowns deploying stuff on a Friday, it is unbelievable to me. It is not even funny any more. It seems cloudflare is held together by marketing only. They should stop all of these stupid initiatives and keep their stack simple. And I'm 100% sure the management responsible for this is already fueling up the ferraris to drive to their beach house. All of us make them rich and they keep on enshittifying the…

If the deployment was related to the React Server issue then maybe it was unavoidable.

Everything related to React is avoidable by not using React.

Re: Cloudflare Down Again – and DownDetector Is Also Down

#140
post #37

Edinburgh Airport is also down, suspending all flights after an "IT issue with our air traffic control provider". Not sure if this is coincidental, but the timing is rather suspicious!

BBC reporting that the airport stated it was unrelated
Post reply on HN