Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare outage on November 18, 2025 post mortem

blog.cloudflare.com

Post reply on HN
Print view View original

191–200 of 953 posts

Re: Cloudflare outage on November 18, 2025 post mortem

#191

Post by SchwKatze »

Rohansi wrote: post #144
SchwKatze wrote: post #111

Earlier quoted context omitted.

Unwrap isn't a synonym for laziness, it's just like an assertion, when you do unwrap() you're saying the Result should NEVER fail, and if does, it should abort the whole process. What was wrong was the developer assumption, not the use of unwrap.

> when you do unwrap() you're saying the Result should NEVER fail Returning a Result by definition means the method can fail.

Yeah, I think I expressed wrongly here. A more correct version would be: "when you do unwrap() you're saying that an error on this particular path shouldn't be recoverable and we should fail-safe."

Re: Cloudflare outage on November 18, 2025 post mortem

#192

Post by kloop »

SchemaLoad wrote: post #131
SchwKatze wrote: post #111

Earlier quoted context omitted.

Unwrap isn't a synonym for laziness, it's just like an assertion, when you do unwrap() you're saying the Result should NEVER fail, and if does, it should abort the whole process. What was wrong was the developer assumption, not the use of unwrap.

It also makes it very obvious in the code, something very dangerous is happening here. As a code reviewer you should see an unwrap() and have alarm bells going off. While in other languages, critical errors are a lot more hidden.

I hate that it's a method. That can get lost in a method chain easily enough during a code review.

A function or a keyword would interrupt that and make it less tempting

Re: Cloudflare outage on November 18, 2025 post mortem

#193

Post by cherryteastain »

gpm wrote: post #181
cvhc wrote: post #122

Earlier quoted context omitted.

Some languages and style guides simply forbid throwing exceptions without catching / proper recovery. Google C++ bans exceptions and the main mechanism for propogating errors is `absl::Status` which the caller has to check. Not familiar with Rust but it seems unwrap is such a thing that would be banned.

Unwrap is used in places where in C++ you would just have undefined behavior. It wouldn't make any more sense to blanket ban it than it would ban ever dereferencing a pointer just in case its null - even if you just checked that it wasn't null.

Rust's Result is the same thing as C++'s std::expected. How is calling std::expected::value undefined behaviour?

Re: Cloudflare outage on November 18, 2025 post mortem

#195

Post by shadowgovt »

ojosilva wrote: post #42

This is the multi-million dollar .unwrap() story. In a critical path of infrastructure serving a significant chunk of the internet, calling .unwrap() on a Result means you're saying "this can never fail, and if it does, crash the thread immediately."The Rust compiler forced them to acknowledge this could fail (that's what Result is for), but they explicitly chose to panic instead of handle it gracefully. This is text…

In addition, it looks like this system wasn't on any kind of 1%/10%/50%/100% rollout gating. Such a rollout would trivially have shown the poison input killing tasks.

Re: Cloudflare outage on November 18, 2025 post mortem

#196

Post by Dylan16807 »

Rohansi wrote: post #188
Dylan16807 wrote: post #172

Earlier quoted context omitted.

Some call points to a function that returns a Result will never return an Error. Some call points to a function that returns an int will never return -2. Sometimes you know things the type system does not know.

The difference is functions which return Result have explicitly chosen to return a Result because they can fail. Sure, it might not fail in the current implementation and/or configuration, but that could change later and you might not know until it causes problems. The type system is there to help you - why ignore it?

Because it would be a huge hassle to go into that library and write an alternate version that doesn't return a Result. So you're stuck with the type system being wrong in some way. You can add error-handling code upfront but it will be dead code at that point in time, which is also not good.

Re: Cloudflare outage on November 18, 2025 post mortem

#197

Post by dist1ll »

tux3 wrote: post #165
dist1ll wrote: post #149

Earlier quoted context omitted.

> every unwrap in production code needs an INFALLIBILITY comment. clippy::unwrap_used can enforce this. How about indexing into a slice/map/vec? Should every `foo[i]` have an infallibility comment? Because they're essentially `get(i).unwrap()`.

Usually you'd want to write almost all your slice or other container iterations with iterators, in a functional style. For the 5% of cases that are too complex for standard iterators? I never bother justifying why my indexes are correct, but I don't see why not. You very rarely need SAFETY comments in Rust because almost all the code you write is safe in the first place. The language also gives you the tool to avoid…

For iteration, yes. But there's other cases, like any time you have to deal with lots of linked data structures. If you need high performance, chances are that you'll have to use an index+arena strategy. They're also common in mathematical codebases.

Re: Cloudflare outage on November 18, 2025 post mortem

#198

Post by alhirzel »

binarymax wrote: post #3

28M 500 errors/sec for several hours from a single provider. Must be a new record. No other time in history has one single company been responsible for so much commerce and traffic. I wonder what some outage analogs to the pre-internet ages would be.

> I wonder what some outage analogs to the pre-internet ages would be.

Lots of things have the sky in common. Maybe comet-induced ice ages...

Re: Cloudflare outage on November 18, 2025 post mortem

#199

Post by eastdakota »

philipgross wrote: post #174
eastdakota wrote: post #95

Earlier quoted context omitted.

Well… we have a culture of transparency we take seriously. I spent 3 years in law school that many times over my career have seemed like wastes but days like today prove useful. I was in the triage video bridge call nearly the whole time. Spent some time after we got things under control talking to customers. Then went home. I’m currently in Lisbon at our EUHQ. I texted John Graham-Cumming, our former CTO and current…

You call this transparency, but fail to answer the most important questions: what was in the burrito? Was it good? Would you recommend?

Chicken burrito from Coyo Taco in Lisbon. I am not proud of this. It’s worse than ordering from Chipotle. But there are no Chipotle’s in Lisbon… yet.

Re: Cloudflare outage on November 18, 2025 post mortem

#200

Post by pyrolistical »

nomel wrote: post #178
yoyohello13 wrote: post #163

Earlier quoted context omitted.

Probably not, since errors as values are way better than exceptions.

How so? An exception is a value that's given the closest, conceptually appropriate, point that was decided to handle the value, allowing you to keep your "happy path" as clean code, and your "exceptional circumstances" path at the level of abstraction that makes sense. It's way less book-keeping with exceptions, since you, intentionally, don't have to write code for that exceptional behavior, except where it makes se…

Exception is hidden control flow, where as error values are not.

That is the main reason why zig doesn’t have exceptions.

Post reply on HN