Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

10% of Firefox crashes are caused by bitflips

mas.to

Post reply on HN
Print view View original

341–350 of 495 posts

Re: 10% of Firefox crashes are caused by bitflips

#341

Post by asimovDev »

ordu wrote: post #219
newscracker wrote: post #183

This is quite surprising to me, since I thought the percentage would be a lot lesser. But I don’t really know what the Firefox team does with crash reports and in making Firefox almost crash proof. I have been using it at work on Windows and for the last several years it always crashes on exit. I have religiously submitted every crash report. I even visit the “about:crashes” page to see if there are any unsubmitted o…

It is hard to judge, but a crash on exit seems to me a possible consequence of a damaged memory. Firefox frees all the resources and collects the garbage. I expect it to touch a lot of memory locations, and do something with values retrieved. > this crash-on-exit happens due to many different causes, as seen in the crash reports It points to the same direction: all these different causes are just symptoms, the root c…

Surely hardware issues would manifest in other software or overall OS as well?

Re: 10% of Firefox crashes are caused by bitflips

#342

Post by VorpalWay »

rafaelmn wrote: post #303
throwaway85825 wrote: post #182

Earlier quoted context omitted.

Very few applications scale with cores. For the vast majority of people single core performance is all they care about, it's also cheaper. They don't need or want workstation gear.

> Very few applications scale with cores You mean like compilers and test suites ? Very few professional workloads don't parallelize well these days.

Compilers and test suits do scale (at least for C/C++ and Rust, which is what I work with). But I think the parent comment referred to consumer applications: games, word processing, light browsing, ...

(Though games these days scale better than they used to, but only up to a to a point.)

I find that most tools I write for my own use can be made to scale with cores, or run so fast that the overhead of starting threads is longer than the program runtime. But I write that in Rust which makes parallelism easy. If I wrote that code in C++ I would probably not bother with trying to parallelize.

Re: 10% of Firefox crashes are caused by bitflips

#343

Post by bjourne »

compiler-guy wrote: post #150
bjourne wrote: post #71

Earlier quoted context omitted.

IME, random bitflips is the engineer's way of saying "I'm sick and tired of root cause analysis" or "I have no fucking clue what the bug is." I, like others, remain skeptical about the claim.

“I have no data, but I’m sure those who do have data, and have spent a significant amount of time analyzing it, are wrong.”

Well, touché. But I'm willing to change my mind once I've seen that data and the methodology Svelto used to analyze it. Extraordinary claims require extraordinary evidence.

Re: 10% of Firefox crashes are caused by bitflips

#344

Post by close04 »

Agingcoder wrote: post #248
Helmut10001 wrote: post #207

Earlier quoted context omitted.

I don't understand why ECC memory is not the norm these days. It is only slightly more expensive, but solves all these problems. Some consumer mainboards even support it already.

No it doesn’t :-) I’ve had plenty of servers with faulty ecc dimms that didn’t trigger , and would only show faults when actual memory testing. I had a hard time convincing some of our admins the first time ( ‘no ecc faults you can’t be right ‘ ) but I won the bet. Edit: very old paper by google on these topics. My issues were 6-7 years ago probably. https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

If we’re being pragmatic, it solves enough problems that you could still call it an undisputed win for stability.

Re: 10% of Firefox crashes are caused by bitflips

#345

Post by zinekeller »

c-c-c-c-c wrote: post #274
adonovan wrote: post #198

Earlier quoted context omitted.

Good point: I-cache is memory too. (Indeed it is SRAM, so its bits might be even more fragile than DRAM!)

Why would a 6T cell (SRAM) be more fragile than a 1T1C (DRAM) cell?

Because it's SRAM, and therefore it still can lose its electrons because we're working with cells a few atoms thick? The loss is not necessarily in L1 (where it's replaced frequently), but in L3 which now has memory comparable to PCs in the early 2000s (and can have its data "stuck" in the same physical area for minutes).

Re: 10% of Firefox crashes are caused by bitflips

#346

Post by ciupicri »

sznio wrote: post #318
Helmut10001 wrote: post #207

Earlier quoted context omitted.

I don't understand why ECC memory is not the norm these days. It is only slightly more expensive, but solves all these problems. Some consumer mainboards even support it already.

What I'm wondering, even without ECC, afaik standard ram still has a parity bit, so a single flip should be detected. With ECC it would be fixed, without ECC it would crash the system. For it to get through and cause an app to malfunction you need two bit flips at least.

I think standard RAM used to have long long time ago, but not anymore. DDR5 finally readd it sort of.

Re: 10% of Firefox crashes are caused by bitflips

#347

Post by gkbrk »

rockdoe wrote: post #245
tredre3 wrote: post #6

> In other words up to 10% of all the crashes Firefox users see are not software bugs, they're caused by hardware defects! If I subtract crashes that are caused by resource exhaustion (such as out-of-memory crashes) this number goes up to around 15%. Crashes caused by resource exhaustion are still software bugs in Firefox. At least on sane operating systems where memory isn't over-comitted.

What's the expected behavior of a JavaScript program that allocates all memory on the machine?

Browser killing the tab way before it happens

Re: 10% of Firefox crashes are caused by bitflips

#348

Post by dmos62 »

vultour wrote: post #293
kleiba wrote: post #246

Firefox is about the only piece of software in my setup that occasionally crashes. I say "occasionally" for lack of a better word, it's not "all the time", but it is definitely more than I would want to. If that was caused by bad memory, I would expect other software to be similarly affected and hence crash with about comparable frequency. However, it looks like I'm falling more into the other 90% of cases (unsurpris…

I spend probably thousands of hours in Firefox every year and I don't think I've ever had it crash.

Same. I don't think I've had a crash in 10+ years.

Re: 10% of Firefox crashes are caused by bitflips

#349

Post by danielEM »

netcoyote wrote: post #33

I've told this story before on HN, but my biz partner at ArenaNet, Mike O'Brien (creator of battle.net) wrote a system in Guild Wars circa 2004 that detected bitflips as part of our bug triage process, because we'd regularly get bug reports from game clients that made no sense. Every frame (i.e. ~60FPS) Guild Wars would allocate random memory, run math-heavy computations, and compare the results with a table of known…

> problems because Dell sourced the absolute cheapest stuff for their computers;

Price itself has nothing to cause problems, it is either bad design or false or incomplete data on datasheets or all of it. Please STOP spreading this narrative, the right thing is to make ads, datasheets, marketing materials etc, etc to tell you the truth that is necessary for you to make proper decision as client/consumer.

Post reply on HN