Live data from Hacker News

Should I buy ECC memory? (2015)

danluu.com

81–90 of 237 posts

Re: Should I buy ECC memory? (2015)

#81

The article makes no mention of single event upsets (SEUs). These occur randomly when cosmic rays can cause a bit flip anywhere in the chip. ECC is a good way to mitigate SEU effects.

Sorry for nitpicking, but it's not the cosmic rays, it's the cosmic rays secondaries cascade shower (produced high up in the atmosphere when a cosmic ray interacts with a particle there).

Re: Should I buy ECC memory? (2015)

#82
post #55

While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…

Google had done extensive studies[1]. There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. However if you are in data center with 100K machines each with 8 DIMM, you are looking at about 6K machines experiencing RAM errors each day . Now if data is being replicated then these errors can propogate corrupted data in unpredic…

That's a 3% per DIMM per year chance of at least one error. Most memory faults are persistent and cause errors until the DIMM is replaced. Also, the error rate was only that low for the smallest DDR2 DIMMs.

Re: Should I buy ECC memory? (2015)

#84
post #72
post #55

Earlier quoted context omitted.

Google had done extensive studies[1]. There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. However if you are in data center with 100K machines each with 8 DIMM, you are looking at about 6K machines experiencing RAM errors each day . Now if data is being replicated then these errors can propogate corrupted data in unpredic…

> There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. How do you make that leap?

Because the market.

Re: Should I buy ECC memory? (2015)

#85
post #67

Earlier quoted context omitted.

Sure, but those are rare and usually include enough budget to include sysadmins and definitely enough to buy ECC memory. For anyone weighing whether ECC is worth it, they are wasting time managing their email server.

Those are not rare at all. Every lawyers office has this problem, every journalist, every banker, every insurance company, every notary public, every administration and so on.

Most of these are not contractually obliged to run their own email servers so whatever problem they have, it's not that specific one.

Re: Should I buy ECC memory? (2015)

#86
post #72
post #55

Earlier quoted context omitted.

Google had done extensive studies[1]. There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. However if you are in data center with 100K machines each with 8 DIMM, you are looking at about 6K machines experiencing RAM errors each day . Now if data is being replicated then these errors can propogate corrupted data in unpredic…

> There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. How do you make that leap?

It's an inappropriate leap. Consumers should have ECC memory too.

However the consumer market has long decided to settle for ECC nowhere and cheap everywhere.

ECC hardware comes at premium option that can easily be +100%. You need support in the memory, the motherboard and the CPU.

Given the price difference, personal computers will have to live with the memory errors. People will not pay double for their computers. Manufacturers will not sacrifice their margin while they can segment the market and make a ton of money off ECC.

Re: Should I buy ECC memory? (2015)

#87
post #72
post #55

Earlier quoted context omitted.

Google had done extensive studies[1]. There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. However if you are in data center with 100K machines each with 8 DIMM, you are looking at about 6K machines experiencing RAM errors each day . Now if data is being replicated then these errors can propogate corrupted data in unpredic…

> There is roughly 3% chance of error in RAM per DIMM per year. That doesn't justify buying ECC if you have just one personal computer to worry about. How do you make that leap?

A cost/benefit analysis for a system where non critical operations are performed would seem to favor the non ECC memory. I suspect this is the case for the majority of people who have computers for their personal use, without taking into account that they might not even be aware such a thing exists. Although, I haven't compared ECC prices lately.

Re: Should I buy ECC memory? (2015)

#88
An old article from DJB worth perusal: http://cr.yp.to/hardware/ecc.html

It's also worth noting that not all ECC (SECDED) is created equal: ChipKill™ and similar might not survive physical damage because of likely shorts of the data bus but a single malfunctioning chip producing/experiencing higher hard error rate is possible from which to recover.

Also, it'd be really cool if some shop a-la BackBlaze blogged about large-scale monitoring for soft and hard RAM errors across chip/module modules (+ motherboards & CPUs). Without collecting and revealing years data from real use, conversation devolves into opinion and conjecture.

Finally, not all use-cases can benefit from ECC (ie Angry Birds) however there are some obvious/nonobvious ones that can (ie router non-ECC DNS bitsquatting or processing bank transactions).

Re: Should I buy ECC memory? (2015)

#89
I've spent the last two weeks looking at Memtest86+ trying to figure out if either one of my memory modules is damaged, or if it is the motherboard. These tests take a long time, and yield different results from day to day.

I've decided to never ever again buy non-ECC memory, at least not on 24/7 servers as well as on workstations.

In a gaming machine / visual typewriter? Sure, non-ECC memory is ok.

Re: Should I buy ECC memory? (2015)

#90
post #9

Depends on what you are doing. ZFS storage servers: Hell yes High-value data in my DB? Hell yes email server: Nope super cool gaming rig: Nope * Cluster: Hell yes General office workstation: maybe. I don't have the budget for 20 redundant copies. I do have the budget for slightly more expensive RAM. Especially on my ZFS storage arrays. ECC memory is like Insurance. You hope you never need it. One real downside that I…

You should be getting MCA (machine check architecture?) notifications in syslog/dmesg if there are ECC correctable errors, and an MCE (machine check exception) on the console for uncorrectable error, based on my experience with SuperMicro xeon servers running FreeBSD. A lot of our servers see a few correctable errors once in a while, and it doesn't affect the usability of the system; but sometimes the number of correctable errors is very high and the system is very sluggish.
Post reply on HN