Everything Is Broken: Shipping Rust-Minidump at Mozilla

Extra shoutouts to the folks at Sentry who also flipped rust-minidump on as their default backend and had to deal with way more exotic issues than I did (and fixed them!) because although Firefox sees some horrendous stuff and gets a bajillion crash reports, it's still one application with one basically stable minidump writing configuration.

They have to deal with basically random apps doing whatever they want and it sounds like hell.

Maybe I'm missing something, but they ported from C++ (because 'C++ is bad donchaknow') to Rust and still ran into problems parsing crash dumps?

If the dump is corrupt then just stop trying to parse/make sense of it; it's garbage.

No we removed many random crashes that the C++ code had. You cannot "simply" discard a crash report if something is slightly off because then you would discard most crash reports. And most debuginfo too.

You can't expect "thing that runs when a process may have just experienced memory corruption" and "all builds of your application for all eternity" and "every toolchain you ever built your program with for all eternity" to be even vaguely reliable, because those things are in the past and we're trying to figure out how to fix the bugs people are experiencing in production today.

It is a horribly miserable answer to tell your coworkers "yeah sorry I know users are getting thousands of crashes this morning but the crash-dumper didn't sign its name in cursive so I'm gonna refuse to let you read the letter it sent at all".

And just an incoherent answer to say "yeah I know this is a stack overflow but it left the stack in a mildly corrupt state so I absolutely refuse to try to even look at the stack and figure anything out about it". Like, that is the entire purpose of a crashreporter, to investigate a program in an invalid state!

This is the excessively fun part of dealing with crash dumps in general. Many of them are going to be 1% corrupt, 99% fine, and somewhere in them likely has vital information about what caused the corruption.

So the entire reason for being for things like rust-minidump are to make enough sense out of files that are known to be corrupt garbage to be able to find bugs.