I would love to see DDR4 vs DDR5 bitflips. As I understand it DDR5 must come with some level of ECC [1]. [1] https://www.corsair.com/us/en/explorer/diy-builder/memory/is...
I wish also for desktop vs laptop ram comparison.
10% of Firefox crashes are caused by bitflips
231–240 of 495 posts
Re: 10% of Firefox crashes are caused by bitflips
#232As someone who has a strong background from hobby projects with five-digit users before going into work, I think one of the most interesting differences I experienced was that the problems you see at scale simply don't exist on small scale projects. Bit flips/bad memory is one of them.
Re: 10% of Firefox crashes are caused by bitflips
#233Earlier quoted context omitted.
That would tell you if there's a bitflip in your test, but not if there's a bitflip in normal program code causing a crash, no? IIUC GP's questions was how do they actually tell after a crash that that crash was caused by a bitflip.
The example I gave in there is of adding sentinel values in your data, so you can check the constants in your data structures later and go "oh, this is overwritten with garbage" versus "oh, this is one or two bits off". I would imagine plumbing things like that through most common structures is what was done there, though I haven't done the archaeology to find out, because Firefox is an enormous codebase to try and f…
But it would be also possible that sentinel value used for comparison changed because of bitflip, not data structure used by program.
Re: 10% of Firefox crashes are caused by bitflips
#234Earlier quoted context omitted.
I haven't seen a single firefox or chrome crash in months now, you should really stress-test your hardware.
I can't recall a single Firefox crash in at least a decade. What are people doing? I run ublock origin, nothing else. I do sometimes have Firefox mobile misbehave where it stops loading new pages and I jave to restart it, but open pages work normally as do all other operations, so not a crash exactly. Happens maybe once a month Edit: more context, I power cycle at least once a week on desktop and the version is typic…
We have 5 computers running Firefox. One computer has regular Firefox crashes. I've done some memory testing that didn't detect anything wrong.
I've tried all kinds of things software-wise but keep getting random crashes.
I wonder if I should do a longer memory test, maybe some CPU stress testing at the same time...
Re: 10% of Firefox crashes are caused by bitflips
#235Hmm, can someone educate me here? Why don't bit flips ever seem to impact the results of calculations in settings like big-data analytics on AWS?
Is it a difference between server hardware managed by knowledgeable people and random hardware thrown together by home PC builders?
Re: 10% of Firefox crashes are caused by bitflips
#236Hmm, can someone educate me here? Why don't bit flips ever seem to impact the results of calculations in settings like big-data analytics on AWS? Is it a difference between server hardware managed by knowledgeable people and random hardware thrown together by home PC builders?
Servers and pro workstations normally have ECC RAM.
Re: 10% of Firefox crashes are caused by bitflips
#237Does anyone know how they can detect hardware defects like this? This sounds like an incredibly hard problem. And I don’t see how they can do this without impacting performance significantly.
Re: 10% of Firefox crashes are caused by bitflips
#238Hmm, can someone educate me here? Why don't bit flips ever seem to impact the results of calculations in settings like big-data analytics on AWS? Is it a difference between server hardware managed by knowledgeable people and random hardware thrown together by home PC builders?
Presumably professional hardware uses ECC memory, which automatically corrects these kinds of errors.
Re: 10% of Firefox crashes are caused by bitflips
#239Earlier quoted context omitted.
I don't understand why ECC memory is not the norm these days. It is only slightly more expensive, but solves all these problems. Some consumer mainboards even support it already.
Bit flips do not only happen inside RAM Also, in a game, there is a tremendously large chance that any particular bit flip will have exactly 0 effect on anything. Sure you can detect them, but one pixel being wrong for 1/60th of a second isn't exactly ... concerning. The chance for a bit flip to affect a critical path that is noticeable by the player is very low, and quite a bit lower if you design your game to react…
> code for radiation hardened environments
I’m aware of code that detects bit flips via unreasonable value detection (“this counter cannot be this high so quickly”). What else is there?
Re: 10% of Firefox crashes are caused by bitflips
#240Earlier quoted context omitted.
It's not even ECC price/availability that bothers me so much, it's that getting CPUs and motherboards that support ECC is non-trivial outside of the server space. The whole consumer class ecosystem is kind of shitty. At least AMD allows consumer class CPUs to kinda sorta use ECC, unlike Intel's approach where only the prosumer/workstation stuff gets ECC.
I've been honestly amazed people actually buy stuff that's not "workstation" gear given IME how much more reliably and consistently it works, but I guess even a generation or two used can be expensive.
There were several years where used cheese grater Mac Pros could be bought and upgraded for very cheap, and were still not too outdated. I only replaced my MacPro4,1 when the M1 mini came out, mainly cause of wattage.