ECC matters • Hacker News Board

Re: ECC matters

#341

Post by buildbuildbuild » Sun, Jan 03, 2021, 8:03 PM UTC

colejohnson66 wrote: post #311

maria_weber23 wrote: post #274

Earlier quoted context omitted.

ECC memory can't eliminate the chances of these failures entirely. They can still happen. Making software resilient against bitflips in memory seems very difficult though, since it not only affects data, but also code. So in theory the behavior of software under random bit flips is well... Random. You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. I…

> You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. The Apollo missions (or was it the Space Shuttle?) did this. They had redundant computers that would work with each other to determine the “true” answer.

This remains common in aerospace, each voting computer is referred to as a "string". https://space.stackexchange.com/questions/45076/what-is-a-fl...

Re: ECC matters

#342

Post by jjeaff » Sun, Jan 03, 2021, 8:04 PM UTC

freeqaz wrote: post #10

I bought ECC RAM for my laptop and it definitely was about 4x the price. It's valuable to me for a few reasons -- peace of mind being a big one. Bit flips happen and are real. I really wish ECC was plentiful and not brutally expensive!

You should be able to check logs for corrected errors, right?

I'm guessing you won't find any.

Re: ECC matters

#343

Post by sobriquet9 » Sun, Jan 03, 2021, 8:04 PM UTC

maria_weber23 wrote: post #274

nostrademons wrote: post #237

I still remember Craig Silverstein being asked what his biggest mistake at Google was and him answering "Not pushing for ECC memory." Google's initial strategy (c. 2000) around this was to save a few bucks on hardware, get non-ECC memory, and then compensate for it in software. It turns out this is a terrible idea, because if you can't count on memory being robust against cosmic rays, you also can't count on the soft…

ECC memory can't eliminate the chances of these failures entirely. They can still happen. Making software resilient against bitflips in memory seems very difficult though, since it not only affects data, but also code. So in theory the behavior of software under random bit flips is well... Random. You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. I…

If you use multiple computers doing the same calculation and then take the answer from the quorum, how do you ensure the computer that does the comparison is not affected by memory failures? Remember that all queries have to through it, so it has to be comparable in scale and power.

Re: ECC matters

#344

Post by sobriquet9 » Sun, Jan 03, 2021, 8:08 PM UTC

slumdev wrote: post #291

maria_weber23 wrote: post #274

Earlier quoted context omitted.

ECC memory can't eliminate the chances of these failures entirely. They can still happen. Making software resilient against bitflips in memory seems very difficult though, since it not only affects data, but also code. So in theory the behavior of software under random bit flips is well... Random. You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. I…

Error-correcting code (the "ECC" in ECC) is just a quorum at the bit level.

Modern error correction codes can do much better than that.

Re: ECC matters

#345

Post by vvanders » Sun, Jan 03, 2021, 8:08 PM UTC

mlyle wrote: post #209

vvanders wrote: post #198

Earlier quoted context omitted.

TDP matters a fair bit in SFF(Small Form Factor) PCs. For instance the 3700x is a fantastic little CPU since it has a 65W TDP but pretty solid performance. In a sandwich style case you're usually limited to low profile coolers like Noctua L9i/L9a since vertical height is pretty limited.

Performance/watt matters. You can just set TDP to what you want with throttling choices. If you want a 45W TDP from the 3700X, you can just pop into Ryzen Master and ask for a 45W TDP. Boom, you're running in that envelope. I think shopping based on TDP is not the best, because it's not comparable between manufacturers and because it's something you can effectively "choose".

Yeah but can I get a metric ton of benchmarks at that 45w setpoint?

I don't really see the reason in paying for a 100w TDP premium if I'm just going to scale it down to 65w.

Re: ECC matters

#346

Post by jgalentine007 » Sun, Jan 03, 2021, 8:09 PM UTC

jjeaff wrote: post #332

jgalentine007 wrote: post #206

Earlier quoted context omitted.

One of the tire pressure sensors in my car tires had a bit flip a couple years ago and I had to reprogram it's ID. Luckily it was a subaru, so only a light came on in the dash. My old Honda crv however would turn traction control on if your pressure was low - which worked by applying brakes to wheels that were slipping. If you were going up a slippery hill you would soon have no power, sliding backwards nearly off th…

How in the world would you ever know that problem was caused by a bit flip and not just one of the countless other reasons that a sensor could fail?

I have a TPMS programming tool (ATEQ QuickSet) and reader (Autel TS401), because I like to swap my winter / summer tires on my own. The TPMS light came on one day and inflating tires didn't help - I used the reader and found that one sensor's ID had changed. When I compared the ID (it was in hex) to the last programming - it was a single bit off. I couldn't reprogram the sensor itself, but I was able to update the ECU with the changed ID using the ATEQ.

I live in Denver but spend a lot of time skiing around 11k feet, maybe the higher elevation means more radiation.

Re: ECC matters

#347

Post by starfallg » Sun, Jan 03, 2021, 8:10 PM UTC

nostrademons wrote: post #237

I still remember Craig Silverstein being asked what his biggest mistake at Google was and him answering "Not pushing for ECC memory." Google's initial strategy (c. 2000) around this was to save a few bucks on hardware, get non-ECC memory, and then compensate for it in software. It turns out this is a terrible idea, because if you can't count on memory being robust against cosmic rays, you also can't count on the soft…

Recent advances have blurred the lines a bit. The ECC memory that we all know and love is mainly side-band EEC, with the memory bus widened to accommodate the ECC bits driven by the memory controller. However as process size shrink, bit flips become more likely to the point that now many types of memory have on-die EEC, where the error correction is handled internally on the DRAM modules themselves. This is present on some DDR4 and DDR5 modules, but information on this is kept internal by the DRAM makers and not usually public.

https://semiengineering.com/what-designers-need-to-know-abou...

There has been a lot of debate regarding this that was summarised in this post -

https://blog.codinghorror.com/to-ecc-or-not-to-ecc/

Re: ECC matters

#348

Post by josefx » Sun, Jan 03, 2021, 8:16 PM UTC

jjeaff wrote: post #333

josefx wrote: post #121

Earlier quoted context omitted.

Just Intel fucking over security by making ECC a non feature on consumer grade hardware - wouldn't be surprised if it was just a single bit flipped in a feature mask.

Well, with as common as a bunch of people in this thread seem to think bit flips are, it should just be a matter of time until that bit gets flipped on your cpu and activates the ecc feature.

That bit probably is either burned in or stored with the firmware in something more permanent than RAM. Modern RAM has the issue that it is optimized for capacity and speed to a point where state changes can leak into nearby bits.

Re: ECC matters

#349

Post by electricshampo1 » Sun, Jan 03, 2021, 8:17 PM UTC

electricshampo1 wrote: post #188

JoeAltmaier wrote: post #92

ECC works if done right. Accessing a memory location can fix bit-flips (ECC is a 'correcting' code). But systems that don't regularly visit every memory location, can accumulate risk. Those dark corners of RAM can eventually get double-bit errors and be uncorrectable. So an OS might 'wash' RAM during idle moments, reading every location in a round-robin manner to get ECC to kick in and auto-correct. Doesn't matter ho…

Patrol scrub is basically this ( https://www.intel.com/content/dam/www/public/us/en/documents... it is built into the memory controller, no OS involvement is needed.

working link:

https://www.intel.com/content/dam/www/public/us/en/documents...

Re: ECC matters

#350

Post by p_l » Sun, Jan 03, 2021, 8:21 PM UTC

jtl999 wrote: post #254

p_l wrote: post #174

Earlier quoted context omitted.

All AMD CPUs with integrated memory controllers support ECC. The CPU also exposes an interface usable by the operating system to verify ECC works - the same interface is used to provide monitoring of memory fault data provided by ECC. They aren't tested on it, so it's possible to get a dud, but it's minuscule chance that isn't worth bothering. Now, to actual issues you can encounter: motherboards The problem is that…

> Another issue is motherboard firmware. Even though AMD supplies the memory init code, the configuration can be tweaked by motherboard vendor, and they might simply break ECC support accidentally (even by something as simple as making a toggle default to false then forgot to expose it in configuration menu). I think some Gigabyte boards are infamous for this in certain circle OTOH: Gigabyte might have a Threadripper…

Gigabyte is also infamous for trying to claim that they implemented UEFI by dropping a build of DUET (UEFI that boots on top of BIOS, used for early development) into BIOS image...