Cloudflare outage on November 18, 2025 post mortem

Post by csomar » Wed, Nov 19, 2025, 8:02 AM UTC

ozgrakkurt wrote: post #459

vsgherzi wrote: post #22

Why does cloudflare allow unwraps in their code? I would've assumed they'd have clippy lints stopping that sort of thing. Why not just match with { ok(value) => {}, Err(error) => {} } the function already has a Result type. At the bare minimum they could've used an expect("this should never happen, if it does database schema is incorrect"). The whole point of errors as values is preventing this kind of thing.... It w…

And the error magically disappears when the function returns it?

Post by z0ltan » Wed, Nov 19, 2025, 8:07 AM UTC

abalone wrote: post #380

ojosilva wrote: post #42

This is the multi-million dollar .unwrap() story. In a critical path of infrastructure serving a significant chunk of the internet, calling .unwrap() on a Result means you're saying "this can never fail, and if it does, crash the thread immediately."The Rust compiler forced them to acknowledge this could fail (that's what Result is for), but they explicitly chose to panic instead of handle it gracefully. This is text…

I’ve led multiple incident responses at a FAANG, here’s my take. The fundamental problem here is not Rust or the coding error. The problem is: 1. Their bot management system is designed to push a configuration out to their entire network rapidly. This is necessary so they can rapidly respond to attacks, but it creates risk as compared to systems that roll out changes gradually. 2. Despite the elevated risk of system…

Post by baq » Wed, Nov 19, 2025, 8:11 AM UTC

dehrmann wrote: post #231

wrs wrote: post #86

Earlier quoted context omitted.

It seems people have a blind spot for unwrap, perhaps because it's so often used in example code. In production code an unwrap or expect should be reviewed exactly like a panic. It's not necessarily invalid to use unwrap in production code if you would just call panic anyway. But just like every unsafe block needs a SAFETY comment, every unwrap in production code needs an INFALLIBILITY comment. clippy::unwrap_used ca…

It's the same blind spot people have to Java's checked exceptions. People commonly resort to Pokemon exception handling and either blindly ignoring or rethrowing as a runtime exception. When Rust got popular, I was a bit confused by people talking about how great Result it's essentially a checked exception without a stack trace.

Post by csomar » Wed, Nov 19, 2025, 8:21 AM UTC

duped wrote: post #218

pdimitar wrote: post #208

While I heavily frown upon using `unwrap` and `expect` in Rust code and make sure to have Clippy tell me about every single usage of them, I also understand that without them Rust might have been seen as an academic curiosity language. They are escape hatches. Without those your language would never take off. But here's the thing. Escape hatches are like emergency exits. They are not to be used by your team to go to…

This is not a reasonable take to me. unwrap/expect are the idiomatic way to express code paths returning Option/Result as unreachable. Bubbling up the error or None does not make the program correct. Panicking may be the only reasonable thing to do. If panicking is guaranteed because of some input mistake to the system your failure is in testing.

Post by BOOSTERHIDROGEN » Wed, Nov 19, 2025, 8:30 AM UTC

jokoon wrote: post #502

I don't understand what's the business of cloudflare. They just sell proxies, to whoever. Why are they the only company doing ddos protection? I just don't get it.