Live data from Hacker News

ECC matters

realworldtech.com

181–190 of 567 posts

Re: ECC matters

#181
post #66
post #33

Earlier quoted context omitted.

Yeah, it's real obnoxious of Intel to silo ECC support off into the Xeon line, isn't it? I switched to ECC memory in 2013 or 2014 with a Xeon E3 (fundamentally a Core i7 without the ECC support fused off) and of course a Xeon-supporting motherboard (with weird "server board" quirks: e.g., no on-board sound device). I love that AMD doesn't intentionally break ECC on its consumer desktop platforms and upgraded to the T…

I've considered using an AMD CPU instead of Intel's Xeon on the primary desktop computer, but even low-end Ryzen Threadripper CPUs have TDP of 180W, which is a bit higher than I'd like. And though ECC is not disabled in Ryzen CPUs, AFAIK it's not tested in (or advertised for) those, so one won't be able to return/replace a CPU if it doesn't work with ECC memory, AIUI, making it risky. Though I don't know how common i…

> I've considered using an AMD CPU instead of Intel's Xeon on the primary desktop computer, but even low-end Ryzen Threadripper CPUs have TDP of 180W, which is a bit higher than I'd like.

Any apples-to-apples comparable Intel CPU will have comparable power use. The difficulty is that Intel didn't really have anything like Threadripper — their i9 series was the most comparable (high clocks and moderate core counts), but i9 explicitly did not support ECC memory, nullifying the comparison.

You're looking at 2950X, probably? That's a Zen+ (previous gen) model. 16 core / 32 thread, 3.5 GHz base clock, launched August 2018.

Comparable Intel Xeon timeline is Coffee Lake at the latest, Kaby lake before that. As far as I can tell, no Kaby Lake nor Coffee Lake Xeons even have 16 cores.

The closest Skylake I've found is an (OEM) Xeon Gold 6149: 16/32 core/thread, 3.1 GHz base clock, 205W nominal TDP (and it's a special OEM part, not available for you). The closest buyable part is probably Xeon Gold 6154 with 18/36 core/threads, 3GHz clock, and 200W nominal TDP.

Looking at i9 from around that time, you had Skylake-X and a single Coffe Lake-S (i9-9900K). 9900K only has 8 cores. The Skylake i9-9960X part has 16/32 cores/threads, base clock of 3.1GHz, and a nominal TDP of 165W. That's somewhat comparable to the AMD 2950X, ignoring ECC support.

Another note that might interest you: you could run the Threadripper part at substantially lower power by sacrificing a small amount of performance, if thermals are the most important factor and you are unwilling to trust Ryzen ECC: http://apollo.backplane.com/DFlyMisc/threadripper.txt

Or just buy an Epyc, if you want a low-TDP ECC-definitely-supported part: EPYC 7302P has 16/32 cores, 3GHz base clock, and 155W nominal TDP. EPYC 7282 has 16/32 cores, 2.8 GHz base, and 120W nominal TDP. These are all zen2 (vs 2950X's zen+) and will outperform zen+ on a clock-for-clock basis.

> And though ECC is not disabled in Ryzen CPUs, AFAIK it's not tested in (or advertised for) those, so one won't be able to return/replace a CPU if it doesn't work with ECC memory, AIUI, making it risky.

If your vendor won't accept defective CPU returns, buy somewhere else.

> Though I don't know how common it is for ECC to not be handled properly in an otherwise functioning CPU; are there any statistics or estimates around?

ECC support requires motherboard support; that's the main thing to be aware of shopping for Ryzen ECC setups. If the board doesn't have the traces, there's nothing the CPU can do.

Re: ECC matters

#182

Earlier quoted context omitted.

> but even low-end Ryzen Threadripper CPUs have TDP of 180W, which is a bit higher than I'd like. Why does it matter? It doesn't idle that high; it only goes that high of you're using it flat out, in which case the extra power usage is justified because it's giving that much more performance over a 100 W TDP CPU. Now I totally get it if you don't want to go Threadripper just for ECC because it's more expensive , but…

>> I've never seen anyone shop a desktop CPU by TDP, rather than by performance and price. Oh oh, me! Back in the day I bought a 65W CPU for a system that could handle a 90W. I wanted quiet and figured that would keep fan noise down at a modest performance penalty. It should also last longer, being the same design but running cooler. I ran that from 2005 until a few years ago (it still run fine but is in storage). Pl…

On AMD, with Ryzen Master, you can set the TDP-envelope of the processor to what you want. Then the boost/frequency/voltage envelope it chooses to operate in under sustained load is different.

IMO, shopping by performance/watt makes sense. Shopping by TDP doesn't. (Especially since there is no comparing the AMD and Intel TDP numbers as they're defined differently; neither is the maximum the processor can draw, and Intel significantly exceeds the specified TDP on normal workloads).

Re: ECC matters

#183
post #9

I used to be pretty skeptical of ECC for consumer-grade hardware, mainly because I felt that I'd always prefer cheaper/more RAM over ECC RAM even if it meant that I'd get a couple of crash every year due to rogue bitflips. For servers it's a different story, but for a desktop I'm fine dealing with some instability for better performance. But these days with the RAM density being so high and bitflipping attacks being…

Are there any Ryzen boards that support ECC and actually correct errors ?

[deleted]

Re: ECC matters

#184
post #16

I beg this, every time this conversation comes up it’s the same answer “I don’t see a problem”. It’s so easy to chalk these kind of errors to other issues, a little corruption here, a running program goes bezerk there- could be a buggy program or a little accidental memory overwrite. Reboot will fix it. But I ran many thousands of physical machines, petabytes of RAM, I tracked memory flip errors and they were _common…

When you say bitflips were "common" on thousands of physical machines, does that mean you observed thousands of bitflips? Otherwise, I would think that an unlikely event becoming 1000x more likely by sheer numbers would have warped your perception. I believe that hardware reliability is mostly irrelevant, because software reliability is already far worse. It doesn't matter whether a bitflip (unlikely) or some bug (li…

Another comment[1] mentioned 1 bitflip per gigabyte per month. If you have a lot of RAM, that's rather a lot.

> It doesn't matter whether a bitflip (unlikely) or some bug (likely) causes a node to spuriously fail

Except that a bitflip can go undetected. It may crash your software or system, but it also may simply leak errors into your data, which can be far more catastrophic.

[1] https://news.ycombinator.com/item?id=25623206

Re: ECC matters

#185
post #9

I used to be pretty skeptical of ECC for consumer-grade hardware, mainly because I felt that I'd always prefer cheaper/more RAM over ECC RAM even if it meant that I'd get a couple of crash every year due to rogue bitflips. For servers it's a different story, but for a desktop I'm fine dealing with some instability for better performance. But these days with the RAM density being so high and bitflipping attacks being…

Are there any Ryzen boards that support ECC and actually correct errors ?

The functionality seems to all be in the memory controller integrated to the CPU.

Re: ECC matters

#186
post #66
post #33

Earlier quoted context omitted.

Yeah, it's real obnoxious of Intel to silo ECC support off into the Xeon line, isn't it? I switched to ECC memory in 2013 or 2014 with a Xeon E3 (fundamentally a Core i7 without the ECC support fused off) and of course a Xeon-supporting motherboard (with weird "server board" quirks: e.g., no on-board sound device). I love that AMD doesn't intentionally break ECC on its consumer desktop platforms and upgraded to the T…

I've considered using an AMD CPU instead of Intel's Xeon on the primary desktop computer, but even low-end Ryzen Threadripper CPUs have TDP of 180W, which is a bit higher than I'd like. And though ECC is not disabled in Ryzen CPUs, AFAIK it's not tested in (or advertised for) those, so one won't be able to return/replace a CPU if it doesn't work with ECC memory, AIUI, making it risky. Though I don't know how common i…

> And though ECC is not disabled in Ryzen CPUs, AFAIK it's not tested in (or advertised for) those

ECC isn't validated by AMD for AM4 Ryzen models, but it's present and supported if the motherboard also supports it. Many motherboards have ECC support (the manual will say for sure), and a handful of models even explicitly advertise it as a feature.

I have a Ryzen 9 3900X on an ASRock B450M Pro4 and 64 GB of ECC DRAM, and ECC functionality is active and working.

Re: ECC matters

#187
post #26

Earlier quoted context omitted.

The problem is that, if you don't have ECC to detect the errors, it's very hard to know what exactly caused a random, non-reproducible crash. Especially in kernel mode where there's little memory protection and basically any driver could be writing anywhere at any time. I can understand Linus's frustration from that point of view: without ECC RAM when you get some super weird crash report where some pointer got corru…

You could run memtest on a pc without ecc for a couple of days and to estimate the error rate, or not?

Pretty sure most memory test tools like memtest86 write the memory and then read it back shortly thereafter in relatively small blocks. This makes the window for errors to be introduced dramatically smaller. Most memory in a computer is not being continually rewritten under normal use.

Re: ECC matters

#188

ECC works if done right. Accessing a memory location can fix bit-flips (ECC is a 'correcting' code). But systems that don't regularly visit every memory location, can accumulate risk. Those dark corners of RAM can eventually get double-bit errors and be uncorrectable. So an OS might 'wash' RAM during idle moments, reading every location in a round-robin manner to get ECC to kick in and auto-correct. Doesn't matter ho…

Patrol scrub is basically this (https://www.intel.com/content/dam/www/public/us/en/documents... it is built into the memory controller, no OS involvement is needed.

Re: ECC matters

#189
post #33

Earlier quoted context omitted.

Yeah, it's real obnoxious of Intel to silo ECC support off into the Xeon line, isn't it? I switched to ECC memory in 2013 or 2014 with a Xeon E3 (fundamentally a Core i7 without the ECC support fused off) and of course a Xeon-supporting motherboard (with weird "server board" quirks: e.g., no on-board sound device). I love that AMD doesn't intentionally break ECC on its consumer desktop platforms and upgraded to the T…

While it's true that Intel only has ECC support on Xeon (and several other chips targeted at the embedded market) it's not true that ECC is supported well on AMD. We only use Xeons on developer desktops and production machines here precisely because of ECC. It's about 1 bit flip/month/gigabyte. That's too much risk when doing something critical for a client.

> While it's true that Intel only has ECC support on Xeon

That's not true. There are Core i3, Atom, Celeron, and Pentium SKUs with ECC. E.g. the Core i3-9300

https://en.wikichip.org/wiki/intel/core_i3/i3-9300

Re: ECC matters

#190

Earlier quoted context omitted.

That's not necessarily correct. You can passively cool threadrippers if you underclock them enough and have good ventilation in case.

If my only interest would be ECC, I might do that but, I develop scientific software for research purposes. I need every bit of performance from my system. In my case loading means maxing out all cores and extended period of time can be anything from five minutes to hours.

The problem is-- you can't compare the TDP nor even the system cooling design guidelines between AMD and Intel.

Both are optimistic lies, but-- if you look at the documents it looks like currently AMD needs more cooling, but actually dissipates less power in most cases and definitely has higher performance/watt.

Post reply on HN