Should I buy ECC memory? (2015) • Hacker News Board

Re: Should I buy ECC memory? (2015)

#61

Post by boomboomsubban » Wed, Apr 26, 2017, 10:17 PM UTC

ruleabidinguser wrote: post #59

lomnakkus wrote: post #38

Earlier quoted context omitted.

Indeed. It's true that the data may be corrupted before hitting any disk[1], but once it has hit the disks (>1), it's extremely unlikely that you'll ever hit a similar bit error where it'll mistakenly choose the wrong disk block to recover from. The main point of e.g. ZFS or Btrfs checksumming is that a) at least it isn't getting worse , and b) I can tell if it's getting worse. [1] ... but if the bits are not generat…

What are you doing where you're actually checking checksums periodically and detcting when things get worse? That seems like a lot of work to set up.

They are using ZFS, scrubbing is one command.

Re: Should I buy ECC memory? (2015)

#62

Post by throwa38282 » Wed, Apr 26, 2017, 10:17 PM UTC

amenod wrote: post #54

muro wrote: post #34

Earlier quoted context omitted.

I don't understand why anyone would run their own email server. Cloud offerings work so well and are cheap.

NSA agrees with you, for one.

NSA is not the primary threat here. More conventional legal mechanisms are. Home serving is just as vulnerable to the NSA.

Re: Should I buy ECC memory? (2015)

#63

Post by Dylan16807 » Wed, Apr 26, 2017, 10:18 PM UTC

dogma1138 wrote: post #40

mikeash wrote: post #24

Earlier quoted context omitted.

I don't get this association between ZFS and ECC. The recommendation to use ECC with ZFS basically comes down to "all that fancy data integrity checking that ZFS does won't protect you from memory errors, so you'll effectively lose that feature." Are you OK with silent data corruption? If so, don't bother with ECC. If not, use it.

Because with ZFS bit rot can be cumulative, with most file systems a memory error will corrupt a file if the format can't handle errors, with ZFS overtime the entire volume can get corrupted especially when you are doing recovery or expansion, even in normal operation data is moved around quite a bit. For them most part with other common file systems when a file is written it stays there even in RAID.

I don't know, I wouldn't say ZFS moves files around any more than a typical filesystem.

Re: Should I buy ECC memory? (2015)

#64

Post by Prego » Wed, Apr 26, 2017, 10:19 PM UTC

Face it: for the type of work most of you people do--swiping left or right on Grinder, or sitting in a coffee shop pretending to write a screenplay--you don't need ECC memory.

I, however, won't do any production work without it.

Re: Should I buy ECC memory? (2015)

#65

Post by greenshackle2 » Wed, Apr 26, 2017, 10:21 PM UTC

rocqua wrote: post #13

VA3FXP wrote: post #9

Depends on what you are doing. ZFS storage servers: Hell yes High-value data in my DB? Hell yes email server: Nope super cool gaming rig: Nope * Cluster: Hell yes General office workstation: maybe. I don't have the budget for 20 redundant copies. I do have the budget for slightly more expensive RAM. Especially on my ZFS storage arrays. ECC memory is like Insurance. You hope you never need it. One real downside that I…

There is a hidden cost of ECC with regards to the chipset. None of the cheap chipsets support it, so on any home build, it's going to be expensive.

Some older AMD desktop chips support ECC.

I built a home NAS from an old board and Phenom II 545 CPU I had lying around, fortuitously they happen to support ECC. DDR2 unregistered ECC ram was a bit of a pain to find though.

Re: Should I buy ECC memory? (2015)

#66

Post by wtallis » Wed, Apr 26, 2017, 10:21 PM UTC

dogma1138 wrote: post #40

mikeash wrote: post #24

Earlier quoted context omitted.

I don't get this association between ZFS and ECC. The recommendation to use ECC with ZFS basically comes down to "all that fancy data integrity checking that ZFS does won't protect you from memory errors, so you'll effectively lose that feature." Are you OK with silent data corruption? If so, don't bother with ECC. If not, use it.

Because with ZFS bit rot can be cumulative, with most file systems a memory error will corrupt a file if the format can't handle errors, with ZFS overtime the entire volume can get corrupted especially when you are doing recovery or expansion, even in normal operation data is moved around quite a bit. For them most part with other common file systems when a file is written it stays there even in RAID.

The only detailed explanations I've ever seen for how memory errors can snowball into whole-filesystem loss on ZFS have relied on the assumption that you have a deterministically stuck bit in a region of memory that the OS is re-using for different parts of the FS data structures but never anything that could cause the machine itself to crash (thereby clueing you in to a hardware reliability issue).

Do you have a source for a more plausible analysis that takes into account how memory actually tends to fail?

Re: Should I buy ECC memory? (2015)

#67

Post by muro » Wed, Apr 26, 2017, 10:27 PM UTC

jacquesm wrote: post #48

muro wrote: post #34

Earlier quoted context omitted.

I don't understand why anyone would run their own email server. Cloud offerings work so well and are cheap.

Maybe they are under contract not to pass information to third parties or maybe the company policy is to not let internal email off the network. That you don't understand it is likely from the perspective of an individual, possibly a private user. For those applications you can't beat the cloud. For business use every business needs to weigh their own needs. Even then though, many business think they need to have the…

Sure, but those are rare and usually include enough budget to include sysadmins and definitely enough to buy ECC memory. For anyone weighing whether ECC is worth it, they are wasting time managing their email server.

Re: Should I buy ECC memory? (2015)

#68

Post by lomnakkus » Wed, Apr 26, 2017, 10:29 PM UTC

ruleabidinguser wrote: post #59

lomnakkus wrote: post #38

Earlier quoted context omitted.

Indeed. It's true that the data may be corrupted before hitting any disk[1], but once it has hit the disks (>1), it's extremely unlikely that you'll ever hit a similar bit error where it'll mistakenly choose the wrong disk block to recover from. The main point of e.g. ZFS or Btrfs checksumming is that a) at least it isn't getting worse , and b) I can tell if it's getting worse. [1] ... but if the bits are not generat…

What are you doing where you're actually checking checksums periodically and detcting when things get worse? That seems like a lot of work to set up.

zpool scrub

(This may be a myth: It's not something you should actually do that often because actually reading the media may dregrade it.)

Re: Should I buy ECC memory? (2015)

#69

Post by paulsutter » Wed, Apr 26, 2017, 10:31 PM UTC

lomnakkus wrote: post #29

nostrademons wrote: post #7

While I was at Google, someone asked one of the very early Googlers (I think it was Craig Silverstein, but it may've been Jeff Dean) what was the biggest mistake in their Google career, and they said "Not using ECC memory on early servers." If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you could…

> If you look through the source code & postmortems from that era of Google, there are all sorts of nasty hacks and system design constraints that arose from the fact that you couldn't trust the bits that your RAM gave back to you. Details of this would be very interesting, but obviously I understand if you cannot provide such details due to NDAs, etc. I mean, I can imagine a few mitigations (pervasive checksumming,…

You need ECC /and/ pervasive checksumming. There are too many stages of processing where errors can occur. For example, disk controllers or networks. The TCP checksum is a bit of a joke at 16 bits (it will fail to detect 1 in 65000 errors), and even the Ethernet CRC can fail - you need end to end checksums.

http://www.evanjones.ca/tcp-and-ethernet-checksums-fail.html

Re: Should I buy ECC memory? (2015)

#70

Post by blackflame7000 » Wed, Apr 26, 2017, 10:33 PM UTC

static_noise wrote: post #10

godzillabrennus wrote: post #3

Do you use ZFS? If yes then you should use ECC memory. Do you have a use case where you would want your computer to alert you when the ram is failing? If yes then you should use ECC memory. Otherwise it's a nicitey and probably not worth the money.

Do you use ZFS? If no then you should use ECC memory. Now the half truth becomes full-truth.

Here is an article detailing how ZFS is virtually unaffected by random bit flips because it would have to occur in such a way as to cause a sha256 collision between one block and its parity block in order for it to repair a valid block with a corrupt one during a scrub. Furthermore it goes on to argue that only a highly specific large scale ram corruption could possibly cause corruption and by that time it's almost certain the OS wouldn't boot up.

http://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-yo...