Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

10% of Firefox crashes are caused by bitflips

mas.to

Post reply on HN
Print view View original

331–340 of 495 posts

Re: 10% of Firefox crashes are caused by bitflips

#331

Post by samus »

BeetleB wrote: post #90
shevy-java wrote: post #51

> In other words up to 10% of all the crashes Firefox users see are not software bugs, they're caused by hardware defects! Bold claim. From my gut feeling this must be incorrect; I don't seem to get the same amount of crashes using chromium-based browsers such as thorium.

Are people getting so many FF crashes? Mine rarely does. I leave it running, opening and closing tabs, for weeks on end.

It really depends on what you're doing with your hardware. Overclocking, overheating, unstable power supply, and things like that increase the likelihood of memory bitflips.

Re: 10% of Firefox crashes are caused by bitflips

#332

Post by samiv »

netcoyote wrote: post #33

I've told this story before on HN, but my biz partner at ArenaNet, Mike O'Brien (creator of battle.net) wrote a system in Guild Wars circa 2004 that detected bitflips as part of our bug triage process, because we'd regularly get bug reports from game clients that made no sense. Every frame (i.e. ~60FPS) Guild Wars would allocate random memory, run math-heavy computations, and compare the results with a table of known…

Plot twist. The memory bit flip checking code was actually buggy and contained UB.

No, seriously did you actually verify the code for correctness before relying on it's results?

Re: 10% of Firefox crashes are caused by bitflips

#333

Post by matja »

pulkas wrote: post #257

what happens if bitflip occurs while you are detecting bitflip? bitflippin...

I have a machine with a 6 year uptime that was slowly accumulating single bit error corrections. The EDAC counter mysteriously stopped at 308 last year, and hasn't changed since, so I wonder if a bitflip in the counter circuit made it stop...

Re: 10% of Firefox crashes are caused by bitflips

#337

Post by meindnoch »

sznio wrote: post #318
Helmut10001 wrote: post #207

Earlier quoted context omitted.

I don't understand why ECC memory is not the norm these days. It is only slightly more expensive, but solves all these problems. Some consumer mainboards even support it already.

What I'm wondering, even without ECC, afaik standard ram still has a parity bit, so a single flip should be detected. With ECC it would be fixed, without ECC it would crash the system. For it to get through and cause an app to malfunction you need two bit flips at least.

Wrong. Regular RAM has no parity bit.

Re: 10% of Firefox crashes are caused by bitflips

#338

Post by pizza »

d1sxeyes wrote: post #264
gmueckl wrote: post #255

Earlier quoted context omitted.

For safety critical systems, one strategy is to store at least two copies of important data and compare them regularly. If they don't match, you either try to recover somehow or go into a safe state, depending on the context.

At least three copies, so you can recover based on consensus.

“never go to sea with two chronometers, take one or three”

Re: 10% of Firefox crashes are caused by bitflips

#339

Post by nxobject »

netcoyote wrote: post #33

I've told this story before on HN, but my biz partner at ArenaNet, Mike O'Brien (creator of battle.net) wrote a system in Guild Wars circa 2004 that detected bitflips as part of our bug triage process, because we'd regularly get bug reports from game clients that made no sense. Every frame (i.e. ~60FPS) Guild Wars would allocate random memory, run math-heavy computations, and compare the results with a table of known…

> Several years later I learned that Dell computers had larger-than-reasonable analog component problems because Dell sourced the absolute cheapest stuff for their computers; I expect that was also a cause

Oh god yes… Dell OptiPlexes and bad caps went together in those days. I’m half convinced Valve put the gray towers in Counter-Strike so IT employees wasting time could shoot them up for therapy.

Re: 10% of Firefox crashes are caused by bitflips

#340

Post by monadgonad »

shiroiuma wrote: post #27
kmoser wrote: post #7

The next logical step would be to somehow inform users so they could take action to replace the bad memory. I realize this is a challenge given the anonymized nature of the crash data, but I might be willing to trade some anonymity in exchange for stability.

>The next logical step would be to somehow inform users so they could take action to replace the bad memory. This isn't really feasible: have you looked at memory prices lately? The users can't afford to replace bad memory now.

The current situation really has zero bearing on the principle that it’s better to inform users of this.
Post reply on HN