21–30 of 237 posts
Altitude also plays a factor in random memory corruption. From the wikipedia article on ECC Ram, "Hence, the error rates increase rapidly with rising altitude; for example, compared to the sea level, the rate of neutron flux is 3.5 times higher at 1.5 km and 300 times higher at 10–12 km (the cruising altitude of commercial airplanes).[3] As a result, systems operating at high altitudes require special provision for r…
Depends on what you are doing. ZFS storage servers: Hell yes High-value data in my DB? Hell yes email server: Nope super cool gaming rig: Nope * Cluster: Hell yes General office workstation: maybe. I don't have the budget for 20 redundant copies. I do have the budget for slightly more expensive RAM. Especially on my ZFS storage arrays. ECC memory is like Insurance. You hope you never need it. One real downside that I…
There is a hidden cost of ECC with regards to the chipset. None of the cheap chipsets support it, so on any home build, it's going to be expensive.
Yes. Bit errors are uncommon and range from benign to crash. Your storage has them, memory has them, network has them. Non error correcting memory very significantly increases risk. And this is the kind of risk you don't notice, until you do and when you do, it's often subtle, insidious, impossible to track down. Servers absolutely. It's debatable on desktop, but we have huge RAM now. Might as well error correct. The…
ECC is costly. The memory modules itself and the board required to support it properly.
Do you use ZFS? If yes then you should use ECC memory. Do you have a use case where you would want your computer to alert you when the ram is failing? If yes then you should use ECC memory. Otherwise it's a nicitey and probably not worth the money.
Do you use ZFS? If no then you should use ECC memory. Now the half truth becomes full-truth.
Are you OK with silent data corruption? If so, don't bother with ECC. If not, use it.
While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…
Yes. Bit errors are uncommon and range from benign to crash. Your storage has them, memory has them, network has them. Non error correcting memory very significantly increases risk. And this is the kind of risk you don't notice, until you do and when you do, it's often subtle, insidious, impossible to track down. Servers absolutely. It's debatable on desktop, but we have huge RAM now. Might as well error correct. The…
I won't bother on a desktop. I've been using 4 machines for the last 17 years with storage varying from 10GB to 2TB and RAM varying from 128MB to 16GB and haven't personally seen any kind of data corruption in motion (or at rest for that matter). Only had 2 mechanical drives fail (though predictably). ECC is costly. The memory modules itself and the board required to support it properly.
Yes. Bit errors are uncommon and range from benign to crash. Your storage has them, memory has them, network has them. Non error correcting memory very significantly increases risk. And this is the kind of risk you don't notice, until you do and when you do, it's often subtle, insidious, impossible to track down. Servers absolutely. It's debatable on desktop, but we have huge RAM now. Might as well error correct. The…
I won't bother on a desktop. I've been using 4 machines for the last 17 years with storage varying from 10GB to 2TB and RAM varying from 128MB to 16GB and haven't personally seen any kind of data corruption in motion (or at rest for that matter). Only had 2 mechanical drives fail (though predictably). ECC is costly. The memory modules itself and the board required to support it properly.
Ever had a program crash, hang, or act oddly? That's how data corruption in memory surfaces.
Of course, non-perfect programs (i.e. all of them) act the same way, which means that differentiating memory corruption from misbehaving programs is hard.
Fixing the memory errors will result in more stable system, but it still won't be perfect.
While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…
Details of this would be very interesting, but obviously I understand if you cannot provide such details due to NDAs, etc.
I mean, I can imagine a few mitigations (pervasive checksumming, etc), but ultimately there's very little you can actually do reliably if your memory is lying to you[1]. I can imagine that probabilistic programming would be an option, but it's hardly "mainstream" nor particularly performant :)
I'm also somewhat dismayed at the price premium that Intel are charging for basic ECC support. This is a case where AMD really is a no-brainer for commodity servers unless you're looking for single-CPU performance.
[1] Incidentally also true of humans.
Earlier quoted context omitted.
Do you use ZFS? If no then you should use ECC memory. Now the half truth becomes full-truth.
I don't get this association between ZFS and ECC. The recommendation to use ECC with ZFS basically comes down to "all that fancy data integrity checking that ZFS does won't protect you from memory errors, so you'll effectively lose that feature." Are you OK with silent data corruption? If so, don't bother with ECC. If not, use it.
If you implement ZFS for that propose and cheap out on RAM, you're at odds for that purpose.