Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-ECC memory corrupted my hard drive image [video]

youtube.com

Post reply on HN
Print view View original

71–80 of 161 posts

Re: Non-ECC memory corrupted my hard drive image [video]

#71

Post by eternityforest »

simoncion wrote: post #63
AdrianB1 wrote: post #49

Earlier quoted context omitted.

It is called DDR5 - it has ECC built in the module itself. Making such a module does not make much sense if you cannot report the rate of errors, so if it is just hiding you have a bad RAM stick there is only so much value in having ECC.

It is my understanding that the ECC that you're talking about only protects data-in-flight between the module and whatever is reading or writing the data. It does not protect against corruption of data-at-rest, which is what is protected with ECC in DDR4 and older. It's also my understanding that the DDR5 data-in-flight ECC is a mandatory feature because the link between the memory modules and everything else is so e…

Taking a quick glace at the articles, I think it's the opposite, DDR5 protects data at rest only, because they want to make the chips so unreliable it can't work without it, not the bus.

But in practice, it will probably be more reliable than DDR4 without ECC, since now you need 2 cosmic ray flips, or 1 plus a manufacturing defect flip, and the defect flips will probably be uncommon-ish.

It's too bad data in flight isn't protected without old fashioned ECC on top of that, but it will probably be a big step up, the same way that flash memory is now very reliable even though the actual uncorrected errors are probably worse under the hood.

Re: Non-ECC memory corrupted my hard drive image [video]

#72

Post by IYasha »

moloch-hai wrote: post #9

We don't have ECC mainly because Intel has long been hostile to "consumer" access to ECC. Apparently this was conceived as a market segmentation scheme: people outfitting servers could get ECC when they pay a huge premium. They would thereby not be tempted to cheap out and buy consumer-grade equipment, otherwise wholly adequate to meet all their needs at a radically cheaper price. That we cannot get laptops or even d…

For this reason for my first truly made-from-scratch home NAS I went AMD64 with ECC UDIMMs. It was some very basic Athlon64, but it COULD do ECC. Since then I moved to Opterons and Xeons but I still remember that choice.

Re: Non-ECC memory corrupted my hard drive image [video]

#73

Post by IYasha »

dale_glass wrote: post #38
vladvasiliu wrote: post #34

Earlier quoted context omitted.

> Since then I'm very paranoid, always run memtest on any new RAM I buy overnight, and have ECC where it's possible to have it. Yeah, I do the same, but I've learned that you have to do it regularly. In one of my desktop machines, the RAM ran fine for like two years. Then, all of sudden, random Firefox segfaults, etc. Whipped up a memtest ISO, and sure enough, one of the sticks was bad.

That's the nice thing about ECC, it acts like an always running memory test. You normally have a scrub time that can be configured in the BIOS, which also adds a regular verification of the entire RAM at regular intervals, just in case something goes wrong in some rarely used part of the memory.

Unfortunately, background scrubbing significantly increases power consumption and impacts performance as well.

Re: Non-ECC memory corrupted my hard drive image [video]

#74

Post by IYasha »

eternityforest wrote: post #40
OJFord wrote: post #36

Earlier quoted context omitted.

I realise that's blurrier when it comes to laptops, but AIUI it's more a case of whether the motherboard than supports it than about the AMD chip. i.e. given a desktop CPU, as far as I know you can put it in a motherboard that either does or does not support ECC RAM.

I'm surprised nobody has made RAM with the ECC logic built into the ram itself, that just looks like normal ram to the CPU.

Having ECC being checked inside the CPU is actually useful as data loss may be induced by EMI (and other factors) on PCB data lines.

Re: Non-ECC memory corrupted my hard drive image [video]

#75

Post by dmitrybrant »

justsomehnguy wrote: post #8
TacticalCoder wrote: post #2

> Why ECC Memory Is So Important Except that it's not for many use cases. It's great for servers but for people on their personal and/or work computer, it's simply not that useful. Seriously: which percentage of developers have ECC on their development machine(s)? As developers we live in a world of SSH, cryptographic hashes, checksums everywhere, Git repositories (that is a big one), Merkle trees, digital signatures…

And condoms are only 99.8% effective, so let's not use it? You can always pull out in time or verify checksu^W^W Plan B? > But really most of all I live in a world of Git repositories That's great for you , but 99% don't even know what Git is, just like checksums, cryptographic hashes, Merkle trees, digital signatures and reproducible builds . You just miss the one imortant thing: ECC isn't that helpful where you hav…

Hang on... to go from "System32" to "System33" is one bit flip, but to go from "System33" to "System34" is three bit flips at once. Doesn't that seem astronomically more likely to be a malicious program than bad RAM?

Re: Non-ECC memory corrupted my hard drive image [video]

#76

Post by gmokki »

ksec wrote: post #26

This is especially true for NAS. And why you need BTRFS or preferably ZFS. Unfortunately none of the consumer NAS offers ZFS, and BTRFS is still not a default option. Neither Synology or Qnap seems to care.

I had RAM go bad after running 18T (5 HDDs in raid1) btrfs system in closet for years. Btrfs of course noticed it and fixed most of them automatically when some of blocks were corrupt. But eventually the system failed: the tree that contains the checksums for all the other trees corrupted itself on both copies of one node. Fixed the HW problem and then had to use hex editor to set the checksum manually to correct value (I modified the kernel to print the expected value). Now the system has been again stable for 3 years.

Re: Non-ECC memory corrupted my hard drive image [video]

#77

Post by metadat »

IYasha wrote: post #73
dale_glass wrote: post #38

Earlier quoted context omitted.

That's the nice thing about ECC, it acts like an always running memory test. You normally have a scrub time that can be configured in the BIOS, which also adds a regular verification of the entire RAM at regular intervals, just in case something goes wrong in some rarely used part of the memory.

Unfortunately, background scrubbing significantly increases power consumption and impacts performance as well.

Do you want (1) a higher rate of stable and correct computations to be performed at a slightly higher energy cost, or (2) a demonstrably less reliable device at a slightly lower energy efficiency?

I'll go for #1 in most cases, as long as the system is to be relied upon for anything deemed important.

Re: Non-ECC memory corrupted my hard drive image [video]

#78

Post by metadat »

aborsy wrote: post #66
ksec wrote: post #26

This is especially true for NAS. And why you need BTRFS or preferably ZFS. Unfortunately none of the consumer NAS offers ZFS, and BTRFS is still not a default option. Neither Synology or Qnap seems to care.

Synology offers ECC in 2023 consumer modes such as 923. Still the experience that synology btrfs provides is nowhere as good as ZFS (due to a lot of limitations).

I bought a 6 disk Synology a few months ago and it came with ECC by default. I did a cursory web search about this just now and ECC support appears to be the norm for 22 (as in the year 2022) model revisions and newer (thankfully!).

Re: Non-ECC memory corrupted my hard drive image [video]

#79

Post by [deleted] »

aborsy wrote: post #66
ksec wrote: post #26

This is especially true for NAS. And why you need BTRFS or preferably ZFS. Unfortunately none of the consumer NAS offers ZFS, and BTRFS is still not a default option. Neither Synology or Qnap seems to care.

Synology offers ECC in 2023 consumer modes such as 923. Still the experience that synology btrfs provides is nowhere as good as ZFS (due to a lot of limitations).

[deleted]

Re: Non-ECC memory corrupted my hard drive image [video]

#80

Post by everybodyknows »

dale_glass wrote: post #12
TacticalCoder wrote: post #2

> Why ECC Memory Is So Important Except that it's not for many use cases. It's great for servers but for people on their personal and/or work computer, it's simply not that useful. Seriously: which percentage of developers have ECC on their development machine(s)? As developers we live in a world of SSH, cryptographic hashes, checksums everywhere, Git repositories (that is a big one), Merkle trees, digital signatures…

> But really most of all I live in a world of Git repositories. My entire Emacs config is versioned under Git (I know YMMV but I like it that way). Some people version under Git their entire user dir. > Tell me how my lack of ECC is going to really make life miserable here? Git will break if RAM is bad just like anything else. That it checksums everything won't save you from checking in corrupt data, the filesystem i…

How trustworthy is git-fsck for detecting random-bit corruption?
Post reply on HN