Anyone remember big iron and servers with uptimes of 5 or 7 nines? I mean: it used to be a thing. Now we have the cloud.
7 9's is 3 seconds of downtime per year. That was never a thing.
Ask HN: Cloudflare Workers are down?
81–87 of 87 posts
Re: Ask HN: Cloudflare Workers are down?
#82Earlier quoted context omitted.
This is usually down to provider implementation which switching the core won't help. The provider controls HTTP calls and errors against the relevant service API. Here are the retries in the provider code https://github.com/hashicorp/terraform-provider-aws/blob/mai... It's hard coded to "certificateCrossServicePropagationTimeout" which is 20 minutes here https://github.com/hashicorp/terraform-provider-aws/blob/mai...
Sure, but Terraform Core doesn't provide any way of getting user feedback in case unexpected situations happen, or aborting while saving the current state, both of which would save me serious amounts of time.
Being able to actually interrupt/cancel would be nice. You can get more feedback by adjusting TF_LOG env var. Logging levels have been getting improvements for a while (it used to just be TRACE that spammed everything)
Re: Ask HN: Cloudflare Workers are down?
#83Earlier quoted context omitted.
Sure, but Terraform Core doesn't provide any way of getting user feedback in case unexpected situations happen, or aborting while saving the current state, both of which would save me serious amounts of time.
Being able to actually interrupt/cancel would be nice. You can get more feedback by adjusting TF_LOG env var. Logging levels have been getting improvements for a while (it used to just be TRACE that spammed everything)
> You can get more feedback by adjusting TF_LOG env var.
yep, but that's often enough useless after the fact. In the PHP world, for example, there's Symfony/Monolog's `fingers_crossed` logger [1]... it keeps logs below the normal threshold in memory, but if there is a single event of a given severity or worse, it dumps out all the logs it has ingested so far for this request.
A real lifesaver that one is.
Re: Ask HN: Cloudflare Workers are down?
#84Earlier quoted context omitted.
Our prod app and staging just completely died. Bad day for somebody at Cloudflare
Our main Marketing website that brings revenue is down. No Sympathy from me. It has been 20 mins now. Losing money as I type this. EDIT: I panicked a little. As a dev, I should have been more sympathetic.
[deleted]
Re: Ask HN: Cloudflare Workers are down?
#85[dead]
Re: Ask HN: Cloudflare Workers are down?
#86Earlier quoted context omitted.
Being able to actually interrupt/cancel would be nice. You can get more feedback by adjusting TF_LOG env var. Logging levels have been getting improvements for a while (it used to just be TRACE that spammed everything)
> You can get more feedback by adjusting TF_LOG env var. yep, but that's often enough useless after the fact. In the PHP world, for example, there's Symfony/Monolog's `fingers_crossed` logger [1]... it keeps logs below the normal threshold in memory, but if there is a single event of a given severity or worse, it dumps out all the logs it has ingested so far for this request. A real lifesaver that one is. [1] https:/…
I used to add TF_LOG_PATH to my shell profile so all TF runs log to disk with a daily cron to truncate the file.
Re: Ask HN: Cloudflare Workers are down?
#87This should be resolved. We’re still investigating the underlying root cause, and intend to share a write-up once we have that in hand. This is not the way we wanted anyone to start their week. (I am the PM lead for Cloudflare Workers: Databases & Storage)
Our public postmortem on the incident: https://blog.cloudflare.com/cloudflare-incident-on-october-3...