Live data from Hacker News

ECC matters

realworldtech.com

141–150 of 567 posts

Re: ECC matters

#141
post #33
post #16

I beg this, every time this conversation comes up it’s the same answer “I don’t see a problem”. It’s so easy to chalk these kind of errors to other issues, a little corruption here, a running program goes bezerk there- could be a buggy program or a little accidental memory overwrite. Reboot will fix it. But I ran many thousands of physical machines, petabytes of RAM, I tracked memory flip errors and they were _common…

Yeah, it's real obnoxious of Intel to silo ECC support off into the Xeon line, isn't it? I switched to ECC memory in 2013 or 2014 with a Xeon E3 (fundamentally a Core i7 without the ECC support fused off) and of course a Xeon-supporting motherboard (with weird "server board" quirks: e.g., no on-board sound device). I love that AMD doesn't intentionally break ECC on its consumer desktop platforms and upgraded to the T…

While it's true that Intel only has ECC support on Xeon (and several other chips targeted at the embedded market) it's not true that ECC is supported well on AMD.

We only use Xeons on developer desktops and production machines here precisely because of ECC. It's about 1 bit flip/month/gigabyte. That's too much risk when doing something critical for a client.

Re: ECC matters

#142
post #98
post #70

Earlier quoted context omitted.

quick search: https://rog.asus.com/forum/showthread.php?112750-List-Asus-M...

Most Ryzen ASRock boards support ECC as well. I'm happily using one right now.

> Most

Circa Zen1 launch, ASRock claimed all of their consumer boards would support ECC.

Re: ECC matters

#143
post #9

I used to be pretty skeptical of ECC for consumer-grade hardware, mainly because I felt that I'd always prefer cheaper/more RAM over ECC RAM even if it meant that I'd get a couple of crash every year due to rogue bitflips. For servers it's a different story, but for a desktop I'm fine dealing with some instability for better performance. But these days with the RAM density being so high and bitflipping attacks being…

Are there any Ryzen boards that support ECC and actually correct errors ?

Yes. E.g., all ASRock boards.

Re: ECC matters

#144
post #16

I beg this, every time this conversation comes up it’s the same answer “I don’t see a problem”. It’s so easy to chalk these kind of errors to other issues, a little corruption here, a running program goes bezerk there- could be a buggy program or a little accidental memory overwrite. Reboot will fix it. But I ran many thousands of physical machines, petabytes of RAM, I tracked memory flip errors and they were _common…

How did you track memory errors across thousands of physical machines?

Re: ECC matters

#145
post #118
post #74

Good news is that for DDR5, ECC is a required part of the spec and should be a feature of every module: https://www.anandtech.com/show/15912/ddr5-specification-rele...

On die ECC is great for increasing reliability, if all else is equal, but if it doesn't report to the memory controller, and if the memory controller doesn't report to the OS, I think it will be worse than status quo, because all else won't be equal. With no feedback, systems are going to continue to run on the edge, but now detectable failures will all be multi-bit; because single bit errors are hidden.

Huh? Why would the memory controller not be updated accordingly? Also I have no idea about Linux or Mac, but Windows has had ECC support and active management for decades?

Re: ECC matters

#146
post #97

Earlier quoted context omitted.

Google is like an evil version of Duck Duck Go.

Nah, to Google is just a generic verb. For example I too do all my googling at Duck Duck Go. Hi alphabet lawyers.

I believe there was a suit against alphabet about this very thing.

They argued that 'Google' has now become a verb meaning 'to search the Internet for' and as such alphabet should have the name taken away.

Re: ECC matters

#147
post #2

“ECC availability matters a lot - exactly because Intel has been instrumental in killing the whole ECC industry with it's horribly bad market segmentation.” Its. There, I finally corrected Linus Torvalds in something. :))

Yeah I'm always annoyed with this kind of mistake. Especially as non-native speakers should know better than the native ones (which usually don't give a f.). Now the point about internally doing ECC is an interesting one, could be a way out of this mess. And apparently ECC is more available in AMD land

I've realized that when I'm engaged in the writing (angry or emotional in some way) I tend to commit more of these mistakes, even though I know the difference between "it's" and "its". Linus is always angry, so that probably makes him commit more orthographic mistakes.

Re: ECC matters

#149
post #16

I beg this, every time this conversation comes up it’s the same answer “I don’t see a problem”. It’s so easy to chalk these kind of errors to other issues, a little corruption here, a running program goes bezerk there- could be a buggy program or a little accidental memory overwrite. Reboot will fix it. But I ran many thousands of physical machines, petabytes of RAM, I tracked memory flip errors and they were _common…

When you say bitflips were "common" on thousands of physical machines, does that mean you observed thousands of bitflips?

Otherwise, I would think that an unlikely event becoming 1000x more likely by sheer numbers would have warped your perception.

I believe that hardware reliability is mostly irrelevant, because software reliability is already far worse. It doesn't matter whether a bitflip (unlikely) or some bug (likely) causes a node to spuriously fail, what matters is that this failure is handled gracefully.

Re: ECC matters

#150
post #20

I don't know if ECC is that important, but reliability of RAM (or any storage) feels pretty crazy to me. 128GB being refreshed every second for a month error requires that the per-bit refresh process has a reliability of 99.9999999999999999% to be flawless. Considering we are dealing with quantum effects (which are inherently probabilistic), I wouldn't trust myself to design anything like that. Now back to ECC, I'll…

RAM is not as reliable as you think. Some ECC memory hardly ever finds an error, some machines see them at a very consistent rate, e.g. 50 errors per TB-day. That would translate to 1-2 errors per day in a 32 GB PC. Without ECC you cannot know in which bucket you are.

If true, that seems like... a very straightforward bucket to test if you’re in.
Post reply on HN