Ugh just write a real blog post dude.
10% of Firefox crashes are caused by bitflips
151–160 of 495 posts
Re: 10% of Firefox crashes are caused by bitflips
#152I've told this story before on HN, but my biz partner at ArenaNet, Mike O'Brien (creator of battle.net) wrote a system in Guild Wars circa 2004 that detected bitflips as part of our bug triage process, because we'd regularly get bug reports from game clients that made no sense. Every frame (i.e. ~60FPS) Guild Wars would allocate random memory, run math-heavy computations, and compare the results with a table of known…
As a mobile dev at YouTube I'd periodically scroll through crash reports associated with code I owned and the long tail/non-clustered stuff usually just made absolutely no sense and I always assumed at least some of it was random bit flips, dodgy hardware, etc.
Re: 10% of Firefox crashes are caused by bitflips
#153> In other words up to 10% of all the crashes Firefox users see are not software bugs, they're caused by hardware defects! Bold claim. From my gut feeling this must be incorrect; I don't seem to get the same amount of crashes using chromium-based browsers such as thorium.
I haven't seen a single firefox or chrome crash in months now, you should really stress-test your hardware.
I can't recall a single Firefox crash in at least a decade. What are people doing? I run ublock origin, nothing else. I do sometimes have Firefox mobile misbehave where it stops loading new pages and I jave to restart it, but open pages work normally as do all other operations, so not a crash exactly. Happens maybe once a month
Edit: more context, I power cycle at least once a week on desktop and the version is typically a bit behind new. I also don't have more tabs open than will fit in the row. All these habits seem likely to decrease crashes.
Re: 10% of Firefox crashes are caused by bitflips
#154When debugging something, I often remember the the quote, often misattributed to Einstein: "Insanity is doing the same thing over and over again and expecting different results". Then I remember about bitflips, and run a second, maybe a third time, just expecting the next bit to flip to not be in the routine I'm trying to debug.
Re: 10% of Firefox crashes are caused by bitflips
#155Re: 10% of Firefox crashes are caused by bitflips
#156I've told this story before on HN, but my biz partner at ArenaNet, Mike O'Brien (creator of battle.net) wrote a system in Guild Wars circa 2004 that detected bitflips as part of our bug triage process, because we'd regularly get bug reports from game clients that made no sense. Every frame (i.e. ~60FPS) Guild Wars would allocate random memory, run math-heavy computations, and compare the results with a table of known…
Mike is such a legend.
Re: 10% of Firefox crashes are caused by bitflips
#157Going to be downvoted, but I call bullshit on this. Bitflips are frequent (and yes ECC is an improvement but does not solve the problem), but not that frequent. One can either assume users that enabled telemetry are an odd bunch with flaky hardware, or the implementation isnt actually detecting bitflips (potentially, as the messages indicate), but a plathora of problems. Having a 1/10 probability a given struct is ei…
How can you possibly be this confident if you don't know the number of times Firefox was run and number of bug reports submitted? Say it's run 100,000,000 times, 1000 reports are submitted, and 10 are bit flips. Seems reasonable. You're misinterpreting what they are saying.
Re: 10% of Firefox crashes are caused by bitflips
#158How many are caused by cosmic radiation bitflips?
An SO question indicates "10 GB of memory should show an ECC event every 1,000 to 10,000 hours,"
https://stackoverflow.com/questions/2580933/cosmic-rays-what...
Re: 10% of Firefox crashes are caused by bitflips
#159Earlier quoted context omitted.
Are people getting so many FF crashes? Mine rarely does. I leave it running, opening and closing tabs, for weeks on end.
Months in my case. But I have ECC. Every five years I build a new development workstation and I always have ECC.
I can also go months and don't see crashes (though occasionally I'll hit a memory leak where closing tabs doesn't release it so I'll restart firefox then), but unless ThinkPads come with ECC I don't have it.
Re: 10% of Firefox crashes are caused by bitflips
#160> In other words up to 10% of all the crashes Firefox users see are not software bugs, they're caused by hardware defects! Bold claim. From my gut feeling this must be incorrect; I don't seem to get the same amount of crashes using chromium-based browsers such as thorium.
...normally browsers don't crash at all. Something's wrong with your computer.