Earlier quoted context omitted.
Totally random.
Could easily be software or some other marginal hardware bug though.
Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
41–50 of 79 posts
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#42ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…
ECC is now baked into the DDR5 spec. Great news!
Additionally the on-chip EEC of DDR5 won't report the errors to your OS. ECC memory errors when corrected can be handled by the OS, and you'll even be informed of the uncorrectable 2 bit errors.
Want to protect for 2-bit errors? Make sure your platform has support for ECC-chipkill.
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#43Bit flips are totally real, at scale you will definitely see them on large queries. There was a fun talk at DEFCON on bitsquatting, the process of buying 1 bit off domain names and then accepting all incoming connections. Attacks like rowhammer similarly abuse erroneous bit flips. Supposedly microsoft can detect solar activity based on the number of windows crash logs they receive. DEFCON Talk: https://www.youtube.co…
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#44I've had a lot of really strange bugs and data loss with my current build (Ryzen with Gskill memory). After running a memtest for 24h i finally saw that two of the four ram sticks were faulty (two bit flips on each only rarely and on a specific test). The company changed them but now a year later without any issues I have another one that failed in exactly the same way. This is the last time I build a non-ECC system…
Gigabyte really did mean DDR4-3200 was the limit for Pinnacle Ridge and older AMD cpus.
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#45ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…
I think you overstate the problem here. Chances are, unless you’ve addressed other more pertinent issues, simply using ECC memory isn’t going to stop systems from crashing in the middle of the night.
Like my desktop just froze, and then it never happened again. It only makes sense if it was a random bit-flip.
The actual RAM speed never mattered, you can't tell the difference with 150 FPS and 165 FPS (even though my screen's refresh rate is 280Hz)
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#46ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…
Intel finally started supporting ECC again on their consumer CPUs, but if only the W680 didn't cost ~$400 starting.
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#47I ran a cluster of ~30k blade based computers booting entirely off iPXE. They didn't have any onboard ssd/disk storage or ECC memory. Every day, a few of them would randomly lock up, they'd reboot with a fresh network image and keep on humming.
How do you even get that many computers without ECC? I think all the blades I've seen have ECC as baseline spec.
https://blog.codinghorror.com/building-a-computer-the-google...
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#48Earlier quoted context omitted.
I think you overstate the problem here. Chances are, unless you’ve addressed other more pertinent issues, simply using ECC memory isn’t going to stop systems from crashing in the middle of the night.
I've had random crashes before that were one-off. Like my desktop just froze, and then it never happened again. It only makes sense if it was a random bit-flip. The actual RAM speed never mattered, you can't tell the difference with 150 FPS and 165 FPS (even though my screen's refresh rate is 280Hz)
10% more fps doesn't matter at 150 fps, but it's nice when your FPS is lower. 60 -> 66 might mean you don't dip below 60 as often. 55 -> 60.5 is pretty nice too. Maybe less of a deal if you've got VRR etc.
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#49ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…
Crashes in the middle of the night are not what worries me. Who cares. It's silent data loss that can go unnoticed for a very long time. And not just a single bit. If the flip hits file system structures or file layout you can have massive silent data loss.
I'd like to think that as NAND continues to scale up in capacity and lower in cost, that we'll see some real shakeup to filesystems and storage where self-healing mass storage can be genuinely commoditized -- not something that's only accessible to businesses (and computing enthusiasts) due to cost and complexity.
Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?
#50Earlier quoted context omitted.
Well, the reason why ECC mattered here is because the RAM was bad, but modern Mac computers do not come with user-serviceable RAM at all, so if you have a problem like this, it's a support ticket anyways, and I'm not even sure there's a true equivalent to Memtest86 for modern Mac computers in the first place. So basically, if it was a RAM problem, there's no point in diagnosing it even if you could; just send your Ma…
ECC is error correcting. A bit gets flipped and it not only detects it but fixes it. Two bits get flipped and it can at least detect it and panic the machine immediately instead of corrupting your data. Without it the corruption is silent. Then this kind of thing happens: https://news.ycombinator.com/item?id=35026440 Which is another reason not to solder the storage either. Suppose you have a system board with bad so…
I did neglect to mention that ECC by-definition can correct errors, but I wonder if what's making people upset with my comment is the implication that ECC can't detect all errors.
But it's true: ECC can't detect all bitflips, and in fact there's at least one study[1] that suggests quite a lot of memory errors go entirely undetected even with ECC.
Silent corruption does in fact occur even with ECC and it may not even be particularly rare, even though it is rarer than typical single/double-bit flips. Of course, the majority of desktops use non-ECC RAM and it's mostly fine, so I assume this is only ever going to matter in production workloads, and exactly what impact it has is hard to gauge.
[1]: https://pages.cs.wisc.edu/~remzi/Classes/739/Fall2018/Papers...