Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Should I buy ECC memory? (2015)

danluu.com

Post reply on HN
Print view View original

121–130 of 237 posts

Re: Should I buy ECC memory? (2015)

#121

Post by fulafel »

blackflame7000 wrote: post #112
mjevans wrote: post #20

A better question is why /shouldn't/ you use ECC memory? Generally the answer to this is any context where you legitimately do NOT care about your data at all, but you still care about costs. This predominately devolves in to consumption only gaming systems. In all other cases everyone would be better served (in the long run) by buying ECC RAM.

a common network topology is to have a load balancer distribute load to a number of cheap Http servers which internally connect to a centralized and powerful database server. In this case only the database server really needs ECC ram. The system is designed to be fault tolerant for any individual HTTP server node so the increased cost vs the problem it solves doesn't make sense. I guess you could argue that a random…

What about memory-error corrupted application data (or application logic) where the corruption occurred on load balancers or web application servers? There's more to data integrity than security holes.

Re: Should I buy ECC memory? (2015)

#122

Post by comex »

derefr wrote: post #78
lomnakkus wrote: post #38

Earlier quoted context omitted.

Indeed. It's true that the data may be corrupted before hitting any disk[1], but once it has hit the disks (>1), it's extremely unlikely that you'll ever hit a similar bit error where it'll mistakenly choose the wrong disk block to recover from. The main point of e.g. ZFS or Btrfs checksumming is that a) at least it isn't getting worse , and b) I can tell if it's getting worse. [1] ... but if the bits are not generat…

> The number of people who religiously check PGP signatures/SHA256sums or whatever is miniscule. • If you transfer things around using BitTorrent, it'll ensure you always end up with a file that hashes correctly to the sum it originally had when the .torrent file was constructed. • Many archive formats (zip, rar, and 7z, at least) contain checksums, and archival utilities validate those checksums during extraction, r…

Both macOS/iOS and Windows use code signing for executables, which should guard against most types of corruption.

Web pages (and anything else) transmitted over HTTPS are protected from corruption in transit by TLS's hashing (which is vastly stronger than the checksums at lower levels of the network stack), though that doesn't help if the server has faulty memory or storage.

PNG has built-in checksums, though other image formats don't (JPEG). Not sure about video.

Re: Should I buy ECC memory? (2015)

#123

Post by nine_k »

blackflame7000 wrote: post #15

Altitude also plays a factor in random memory corruption. From the wikipedia article on ECC Ram, "Hence, the error rates increase rapidly with rising altitude; for example, compared to the sea level, the rate of neutron flux is 3.5 times higher at 1.5 km and 300 times higher at 10–12 km (the cruising altitude of commercial airplanes).[3] As a result, systems operating at high altitudes require special provision for r…

Can adequate shielding help?

Re: Should I buy ECC memory? (2015)

#124

Post by gh02t »

tdb7893 wrote: post #114
sitkack wrote: post #109

Earlier quoted context omitted.

I have hit soft errors in every desktop machine that used ECC. Either I have bad luck, ECC causes the errors or third thing. I think ECC should be mandated for anything except toys and video players.

How much more expensive is ECC ram? I don't have it and I've never experienced obvious issues, if it's a lot more expensive it's not really worth it for the once or twice the desktop will likely experience an actual issue

It's significantly more expensive, usually around 30-100% more, depending on capacity. IMO not worth it on a desktop, possibly worth it on a home server or a serious workstation. Plus your CPU and motherboard has to support it, which is a pain with Intel's consumer lineup.

Re: Should I buy ECC memory? (2015)

#125

Post by m-j-fox »

tdb7893 wrote: post #114
sitkack wrote: post #109

Earlier quoted context omitted.

I have hit soft errors in every desktop machine that used ECC. Either I have bad luck, ECC causes the errors or third thing. I think ECC should be mandated for anything except toys and video players.

How much more expensive is ECC ram? I don't have it and I've never experienced obvious issues, if it's a lot more expensive it's not really worth it for the once or twice the desktop will likely experience an actual issue

Should be about 1/8th more since it's just a 72-bit bus for carrying 64-bits data and 8-bits check. Or rather, your dimm will have 9 chips instead of 8.

How they get you is Intel will sell you a xeon which is the exact same die as an i5 in a different package for more money.

Re: Should I buy ECC memory? (2015)

#126

Post by kscz »

lomnakkus wrote: post #68
ruleabidinguser wrote: post #59

Earlier quoted context omitted.

What are you doing where you're actually checking checksums periodically and detcting when things get worse? That seems like a lot of work to set up.

zpool scrub (This may be a myth: It's not something you should actually do that often because actually reading the media may dregrade it.)

or for btrfs users out there:

btrfs scrub start /mnt/volume_name

Re: Should I buy ECC memory? (2015)

#127

Post by jacquesm »

pvg wrote: post #85
jacquesm wrote: post #76

Earlier quoted context omitted.

Those are not rare at all. Every lawyers office has this problem, every journalist, every banker, every insurance company, every notary public, every administration and so on.

Most of these are not contractually obliged to run their own email servers so whatever problem they have, it's not that specific one.

No, but they are contractually required to keep their customers (and their own) data confidential. And that can lead to them deciding to run their own mailservers as well as other infrastructure. Whether that's a good decision or not is another matter, that mostly depends on execution.

Re: Should I buy ECC memory? (2015)

#128

Post by marcosdumay »

derefr wrote: post #74
lomnakkus wrote: post #29

Earlier quoted context omitted.

> If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you couldn't trust the bits that your RAM gave back to you. Details of this would be very interesting, but obviously I understand if you cannot provide such details due to NDAs, etc. I mean, I can imagine a few mitigations (pervasive checksumming,…

> ultimately there's very little you can actually do reliably if your memory is lying to you 1. Implement everything in terms of retry-able jobs; ensure that jobs fail when they hit checksum errors. 2. if you've got a bytecode-executing VM, extend it to compare its modules to stored checksums, just before it returns from them; and to throw an exception instead of returning if it finds a problem. (This is a lot like M…

How do you calculate those checksums without relying on the memory?

Re: Should I buy ECC memory? (2015)

#129

Post by binarycrusader »

wtallis wrote: post #117
binarycrusader wrote: post #93

Earlier quoted context omitted.

Sorry, I was a little obtuse. What I was inferring was that historically, only higher-priced server chips from Intel had ECC support. In 2013, Intel launched the v3 lower-end Xeon E3s server chips that were closer to the price of the consumer Intel chips and offer ECC with comparable clock speeds. Of course, all of those only have 4 cores instead of 8. Yes, all of the E3s support ecc, but Xeon's didn't always support…

Xeon has implied ECC support for as long as Xeons have had integrated memory controllers, which is just a generation or two further back than the first Xeon E3 product line. Before that, ECC support was a function of the northbridge.

Hmm, perhaps this is a quirk of ark.intel.com then; it shows that (as an example) that the Xeon X5690 supported ECC, but the Xeon L5638 did not.

The list on WikiPedia also seems to imply that not all models did historically, perhaps this reflects the northbridge change?

Re: Should I buy ECC memory? (2015)

#130

Post by Retra »

marcosdumay wrote: post #128
derefr wrote: post #74

Earlier quoted context omitted.

> ultimately there's very little you can actually do reliably if your memory is lying to you 1. Implement everything in terms of retry-able jobs; ensure that jobs fail when they hit checksum errors. 2. if you've got a bytecode-executing VM, extend it to compare its modules to stored checksums, just before it returns from them; and to throw an exception instead of returning if it finds a problem. (This is a lot like M…

How do you calculate those checksums without relying on the memory?

You can't really, but you are now requiring the error to occur specifically in the memory containing your checksum, rather than anywhere in your data.
Post reply on HN