Live data from Hacker News

Cloudflare Network Performance Issues

cloudflarestatus.com

201–210 of 329 posts

Re: Cloudflare Network Performance Issues

#202
post #142
post #72

Good. I'm tired of people not learning that trusting a single gateway with 50% of the internet is bad. Yes, I know, free DDOS protection. There has to be another way of doing this, some mesh based DDOS protection or so.

This seems like a case for putting more of the internet through a single gateway. Having my downtime correlated with everyone else's means users will be more forgiving because they'll perceive it as "the Internet's down" rather than "lkbm's site is broken". (We saw this with CloudFlare. Some users were pissed, and others jumped in with "it's not their fault; AWS is down". That doesn't happen when our stuff specifical…

How cares whose fault it is? In the end, you're not able to provide service to your users and you potentially lose money.

Re: Cloudflare Network Performance Issues

#203

Any workarounds or solutions ? I'm an on-call engineer with lots of questions coming in. I'm not sure what I can do apart from moving the domain off Cloudflare, bug DNS propagation would take a few hours and by then Cloudflare might be up again.

My suggestion is wait. I wouldn't even consider flipping my sites over to another DNS unless the outage begins lasting over four hours or so. A lot of top sites use Cloudflare and this sort of outage is extremely rare for them (I can't remember a time when Cloudflare's own site and dashboard were taken offline).

Then again, it depends on the priority of your site. But there are tons of top sites on Cloudflare and I bet a lot of those places don't have plans for emergency switching over to another DNS provider / CDN on short notice as it's often a fairly disruptive change, especially now that more frontend logic for a site is implemented alongside the CDN/LB.

Re: Cloudflare Network Performance Issues

#204
post #73
post #38

Earlier quoted context omitted.

For the site itself that's totally fine, and the status page is separate as it should be, too.

This is fine-ish... In case of network issues affecting their proxy, being able to change your configuration to allow direct traffic would be really nice.

[deleted]

Re: Cloudflare Network Performance Issues

#205
The Cloudflare DNS-over-HTTPS resolver was serving up 502 errors as well, though the standard port 53 UDP resolver was working. This event definitely made me regret choosing Cloudflare as my sole DoH server.

Re: Cloudflare Network Performance Issues

#206
post #142

Earlier quoted context omitted.

This seems like a case for putting more of the internet through a single gateway. Having my downtime correlated with everyone else's means users will be more forgiving because they'll perceive it as "the Internet's down" rather than "lkbm's site is broken". (We saw this with CloudFlare. Some users were pissed, and others jumped in with "it's not their fault; AWS is down". That doesn't happen when our stuff specifical…

How cares whose fault it is? In the end, you're not able to provide service to your users and you potentially lose money.

Business risk cares. SLAs, SLOs, SLIs, they are all about this; to be able to direct the blame.

Re: Cloudflare Network Performance Issues

#207

Cloudflare: Your status page showed "all systems operational" for over 20 minutes while your primary domain was returning a 502 error. Please change this to update automatically, many other engineering teams depend on you. https://i.imgur.com/qHBM2JW.png

We saw an update within 5 minutes of our sites going down. Are you doing caching somewhere?

Seems unlikely, I don't use cloudflare have never visited the status page before. I noticied several 502 pages this morning and searched for "cloudflare status page" and saw the "all systems operational".

Re: Cloudflare Network Performance Issues

#208

Cloudflare: Your status page showed "all systems operational" for over 20 minutes while your primary domain was returning a 502 error. Please change this to update automatically, many other engineering teams depend on you. https://i.imgur.com/qHBM2JW.png

It was showing correctly for me the first time I checked it.

Re: Cloudflare Network Performance Issues

#209
post #120

Earlier quoted context omitted.

Not necessarily backup records, but you can add multiple A/AAAA records. There is no guaranteed order though. It is possible to have multiple A/AAAA records pointing to different loadbalancers, but I don't know how browsers would deal with this.

The lookup is essentially random. If you point DNS to 2 IP's and one of those goes down, then (without going into detail) half of your requests will fail.

Yes, that is correct.

But it does make it possible for a browser to retry a different IP if a 500 range error, or timeout occurs.

But like I said in my other post: I don't know if browsers actually do this.

Post reply on HN