Relevant post by Jeff Atwood: https://blog.codinghorror.com/to-ecc-or-not-to-ecc/
Should I buy ECC memory? (2015)
51–60 of 237 posts
Re: Should I buy ECC memory? (2015)
#52Yes. Bit errors are uncommon and range from benign to crash. Your storage has them, memory has them, network has them. Non error correcting memory very significantly increases risk. And this is the kind of risk you don't notice, until you do and when you do, it's often subtle, insidious, impossible to track down. Servers absolutely. It's debatable on desktop, but we have huge RAM now. Might as well error correct. The…
Does it make any difference if you're using your desktop to compile stuff?
Re: Should I buy ECC memory? (2015)
#53Yes. Bit errors are uncommon and range from benign to crash. Your storage has them, memory has them, network has them. Non error correcting memory very significantly increases risk. And this is the kind of risk you don't notice, until you do and when you do, it's often subtle, insidious, impossible to track down. Servers absolutely. It's debatable on desktop, but we have huge RAM now. Might as well error correct. The…
I won't bother on a desktop. I've been using 4 machines for the last 17 years with storage varying from 10GB to 2TB and RAM varying from 128MB to 16GB and haven't personally seen any kind of data corruption in motion (or at rest for that matter). Only had 2 mechanical drives fail (though predictably). ECC is costly. The memory modules itself and the board required to support it properly.
This is a bit like an inverse magic stone argument ("This stone repels tigers — How do I know it works? — I'm not seeing any tigers around here, do you?").
Re: Should I buy ECC memory? (2015)
#54Earlier quoted context omitted.
I agree on gaming, but e-mail often contains important information that I wouldn't want to suffer from random corruption.
I don't understand why anyone would run their own email server. Cloud offerings work so well and are cheap.
NSA agrees with you, for one.
Re: Should I buy ECC memory? (2015)
#55While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…
Google had done extensive studies[1]. There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. However if you are in data center with 100K machines each with 8 DIMM, you are looking at about 6K machines experiencing RAM errors each day. Now if data is being replicated then these errors can propogate corrupted data in unpredictable unexplainable way even when there are no bugs in your code! For example, you might encounter your logs containing bad line items which gets aggregated in to report showing bizarre numbers because 0x1 turned in to 0x10000001. You can imagine that debugging this happening every day would be huge nightmare and developers would end up eventually inserting lot of asserts for data consistency all over the places. So ECC becomes important if you have distributed large scale system.
1: http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
Re: Should I buy ECC memory? (2015)
#56While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…
it was indeed Craig
Re: Should I buy ECC memory? (2015)
#57I reproduced this by bit-squatting cloudfront.net after reading about it. So many memory errors!
http://dinaburg.org/bitsquatting.html
Loved the variety as well. Sometimes though requests came to me the Host header was correct!
Re: Should I buy ECC memory? (2015)
#58Earlier quoted context omitted.
Do you use ZFS? If no then you should use ECC memory. Now the half truth becomes full-truth.
I don't get this association between ZFS and ECC. The recommendation to use ECC with ZFS basically comes down to "all that fancy data integrity checking that ZFS does won't protect you from memory errors, so you'll effectively lose that feature." Are you OK with silent data corruption? If so, don't bother with ECC. If not, use it.
> I don't get this association between ZFS and ECC.
Because ZFS was the ONLY file system that would actually catch some memory failures even if you didn't have ECC. So, ZFS got a reputation for being snotty when in reality the hardware it was running on was broken.
Re: Should I buy ECC memory? (2015)
#59Can people here please stop posting that ZFS needs ECC memory. Every filesystem, with any name like FAT, NTFS, EXT4 runs more safe with ECC memory. ZFS is actually one of the few that can still be safer if you don't run with ECC memory. Source: Matthew Ahrens himself: https://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=...
Indeed. It's true that the data may be corrupted before hitting any disk[1], but once it has hit the disks (>1), it's extremely unlikely that you'll ever hit a similar bit error where it'll mistakenly choose the wrong disk block to recover from. The main point of e.g. ZFS or Btrfs checksumming is that a) at least it isn't getting worse , and b) I can tell if it's getting worse. [1] ... but if the bits are not generat…
What are you doing where you're actually checking checksums periodically and detcting when things get worse? That seems like a lot of work to set up.
Re: Should I buy ECC memory? (2015)
#60Can people here please stop posting that ZFS needs ECC memory. Every filesystem, with any name like FAT, NTFS, EXT4 runs more safe with ECC memory. ZFS is actually one of the few that can still be safer if you don't run with ECC memory. Source: Matthew Ahrens himself: https://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=...
In an old discussion regarding ECC/ZFS (in particular, whether hitting bad RAM while scrubbing could corrupt more and more data), user XorNot kindly took a look the ZFS source and wrote
"In fact I'm looking at the RAID-Z code right now. This scenario would be literally impossible because the code keeps everything read from the disk in memory in separate buffers - i.e. reconstructed data and bad data do not occupy or reuse the same memory space, and are concurrently allocated. The parity data is itself checksummed, as ZFS assumes it might be reading bad parity by default."
His full comment can be found here: