Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Network Performance Issues

cloudflarestatus.com

Post reply on HN
Print view View original

291–300 of 329 posts

Re: Cloudflare Network Performance Issues

#291

Post by jgrahamc »

technonerd wrote: post #289
TomAnthony wrote: post #280

They have now released an initial statement [1]: For about 30 minutes today, visitors to Cloudflare sites received 502 errors caused by a massive spike in CPU utilization on our network. This CPU spike was caused by a bad software deploy that was rolled back. Once rolled back the service returned to normal operation and all domains using Cloudflare returned to normal traffic levels. This was not an attack (as some ha…

Ahh the "we test in prod" method

Yeah, except that wasn't meant to be the way things work.

Re: Cloudflare Network Performance Issues

#293

Post by matt_oriordan »

felideon wrote: post #286
matt_oriordan wrote: post #275

Earlier quoted context omitted.

Thanks :)

On the OT subject, the status logo is blurry (at least on a 4k monitor), whereas on the main website it's nice and crisp.

Ok, thanks for the heads up. Will raise a status website issue.

Re: Cloudflare Network Performance Issues

#294

Post by markonen »

adamparsons wrote: post #273
zymhan wrote: post #255

Earlier quoted context omitted.

They reserve the best SLA for Enterprise, naturally. https://www.cloudflare.com/plans/enterprise/ > 100% uptime and 25x Enterprise SLA > In the rare event of downtime, Enterprise customers receive a 25x credit against the monthly fee, in proportion to the respective disruption and affected customer ratio.

Two years missing revenue for enterprise customers, ouch that's going to hurt.

25x the downtime, so about 10 hours' worth of charges if the downtime was 25 minutes.

Re: Cloudflare Network Performance Issues

#295

Post by monkeywork »

adamparsons wrote: post #273
zymhan wrote: post #255

Earlier quoted context omitted.

They reserve the best SLA for Enterprise, naturally. https://www.cloudflare.com/plans/enterprise/ > 100% uptime and 25x Enterprise SLA > In the rare event of downtime, Enterprise customers receive a 25x credit against the monthly fee, in proportion to the respective disruption and affected customer ratio.

Two years missing revenue for enterprise customers, ouch that's going to hurt.

you missed the:

>in proportion to the respective disruption and affected customer ratio.

so no they aren't losing 2 yrs of revenue

Re: Cloudflare Network Performance Issues

#296

Post by MrStonedOne »

dewey wrote: post #244
chaz6 wrote: post #158

Earlier quoted context omitted.

Only if you are a paid customer. The free service does not allow this, and this is why I do not use Cloudflare for personal use.

For personal use you also probably don't need the high availability of switching over your domain the moment they are having problems?

One thing is not every free or pro plan on cloudflare is personal use.

I'm running the web servers, official wiki, and game external resource portal for the most active open source video game on github, through cloudflare, and maybe we might not want our 60 million requests a month website to go down when cloudflare does.

Because I can tell you right now our 300 a month budget (that mind you, is capable of covering 7 game servers that can handle 100 connected players (each)) can't take the 80 dollar hit just to make cloudflare not a single point of failure.

Re: Cloudflare Network Performance Issues

#297

Post by elamje »

It’s better that this happens now than later. I am confident CF will put protections in place to prevent this from happening again, but also put a switch in place to provide an instant fix the next time something like this happens.

It does suck to have a service down for a bit, but what CF offers, at the price point is pretty incredible.

Good luck to CF, and I wish you the best with coming up with a robust future-proof solution.

Re: Cloudflare Network Performance Issues

#298

Post by ilkkao »

TomAnthony wrote: post #280

They have now released an initial statement [1]: For about 30 minutes today, visitors to Cloudflare sites received 502 errors caused by a massive spike in CPU utilization on our network. This CPU spike was caused by a bad software deploy that was rolled back. Once rolled back the service returned to normal operation and all domains using Cloudflare returned to normal traffic levels. This was not an attack (as some ha…

Interesting to hear later if the process was followed. Hard to believe they deploy changes to every production location at once.

Re: Cloudflare Network Performance Issues

#299

Post by RafiqM »

matt_oriordan wrote: post #232
buildbuildbuild wrote: post #152

Cloudflare: Your status page showed "all systems operational" for over 20 minutes while your primary domain was returning a 502 error. Please change this to update automatically, many other engineering teams depend on you. https://i.imgur.com/qHBM2JW.png

Sadly this reminds me of AWS outages too where the same applies. How is it that hundreds of developers know there's an issue before AWS do, or Cloudflare in this instance. See my blog post on similar AWS uptime reporting issues at https://www.ably.io/blog/honest-status-reporting-aws-service . At Ably, our status site had an incident update about Cloudflare issues being worked on (by routing away from CF) before Cloud…

At least in the case with AWS, unfortunately there's business involved - because of their uptime guarantee, incidents that would be called downtime by a purely technical team are left as "operational" or "partly degraded". Otherwise, they might have to shell out millions or tens of millions.

Re: Cloudflare Network Performance Issues

#300

Post by dewey »

MrStonedOne wrote: post #296
dewey wrote: post #244

Earlier quoted context omitted.

For personal use you also probably don't need the high availability of switching over your domain the moment they are having problems?

One thing is not every free or pro plan on cloudflare is personal use. I'm running the web servers, official wiki, and game external resource portal for the most active open source video game on github, through cloudflare, and maybe we might not want our 60 million requests a month website to go down when cloudflare does. Because I can tell you right now our 300 a month budget (that mind you, is capable of covering 7…

If you don’t count this as a personal or hobby project it’s a community project. They don’t have a pricing plan for that so you either have to go pro or go to some other provider who gives you this much for free. Why should a company give you even more pro features for free if you are already getting a lot of things for free?
Post reply on HN