Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ECC matters

realworldtech.com

Post reply on HN
Print view View original

321–330 of 567 posts

Re: ECC matters

#321

Post by DCKing »

CalChris wrote: post #262
loeg wrote: post #33

Earlier quoted context omitted.

Yeah, it's real obnoxious of Intel to silo ECC support off into the Xeon line, isn't it? I switched to ECC memory in 2013 or 2014 with a Xeon E3 (fundamentally a Core i7 without the ECC support fused off) and of course a Xeon-supporting motherboard (with weird "server board" quirks: e.g., no on-board sound device). I love that AMD doesn't intentionally break ECC on its consumer desktop platforms and upgraded to the T…

Yeah, the iMac Pro has the Xeon W and ECC. T'would be nice if the Apple Silicon MacBook Pro had it. There's not much of a reason to pay for the Pro over the Air. But like Linus, I'm going to blame Intel for this situation in the market. Maybe Apple will strike out on its own with Apple Silicon but since their dominant use case is phones, I'll not hold my breath.

Unless something weird happens, the next generation of the Apple M-line will use LPDDR5 memory instead of the LPDDR4X used in the Apple M1. While it probably won't support error correction monitoring, LPDDR5 has built in error correction that silently corrects single bit flips. That alone should be a huge reliability improvement.

LPDDR5 will enable some much needed level of error correction in a metric ton of other future SoC designs too. I look forward to the future Raspberry Pi with built in error correction capabilities.

Re: ECC matters

#322

Post by a1369209993 »

cturner wrote: post #171
cuu508 wrote: post #100

Earlier quoted context omitted.

I haven't seen definite details and test results on these (but haven't looked recently). What specific configurations (CPU, MB, RAM) are known to work? Let's say I have a Ryzen system, how can I check if ECC really works? Like, can I see how many bit flips got corrected in, say, last 24h?

Regarding verification. There is a debian package called edac-utils. As I recall you overclock your RAM and run your system at load in order to generate failures. Looking back at my notes, the output of journalctl -b tells should say something like, "Node 0: DRAM ECC enabled." Then 'edac-ctl --status' should tell you that drivers are loaded. Then you run 'edac-util -v' to report on what it has seen, mc0: 0 Uncorrecte…

> As I recall you overclock your RAM and run your system at load in order to generate failures.

You can also use memtest86+ for this, although I don't recall if it requires specific configuration for ECC testing.

Re: ECC matters

#323

Post by Springcleaning »

cbanek wrote: post #77

As someone who has had to read thousands of random game crash reports from all over the interwebs (you know when Windows says you might want to send that crash log? like that), I totally agree. Of all the things to be worried about, like OS bugs, bad hardware configuration, etc. bad memory is one of those really troubling things. You look at the code and say "it's can't make it here, because this was set" but when yo…

Worse than a game crash is your data.

It is incomprehensible that there are still NAS devices being sold without ECC support.

Synology took a step in the right direction to offer prosumer devices with ECC but it is not really advertised as such. It is actually difficult to find which do have ECC and which ones don't.

Re: ECC matters

#324

Post by toast0 »

FartyMcFarter wrote: post #139

Does anyone know why ECC memory requires the CPU to support it? Naively, I can understand why error reporting has dependencies on other parts of the system, but it would seem possible for error correction to work transparently.

As implemented today, ECC is a feature of the memory controller. You need special ram, because instead of 8 parallel rams per bank, you need 9, and all the extra data lines to go to the controller.

Modern CPUs have integrated memory controllers, so that's why the CPU needs to support it.

Correction without reporting isn't great; anyway, you need a reporting mechanism for uncorrectable errors, or all you've done is ensure any memory errors you do experience are worse.

Re: ECC matters

#325

Post by dancek »

jacquesm wrote: post #226
jankeymeulen wrote: post #119

Earlier quoted context omitted.

Or Swedish for that matter, as I believe Torvalds maternal language is Swedish

Finnish is stupendously hard. Far harder than Swedish, at least, by my estimation.

Yes. Swedish is also easy compared to English and French, the other two languages I've learned after early childhood. The only thing that makes it hard is that you never really have use for it and you're forced to learn it nevertheless here in Finland.

I'm happy to see people here on HN respect the difficulty of learning languages. Most foreigners that speak Finnish do it very poorly at first and even after decades they still sound like foreigners. But it shows huge respect to our small country for someone to make the effort, and we really appreciate it. I'm hoping other people see learning their own mother tongue the same way. Sure, most of us need English, but learning it well is still a huge task.

Re: ECC matters

#326

Post by protomolecule »

raverbashing wrote: post #23
simias wrote: post #13

Earlier quoted context omitted.

For a 2nd language speaker making these homophonic mistakes is actually a sign of fluency. It means that you just transcribe a mental flow of words instead of consciously constructing the language. The first time I wrote "your" instead of "you're" in English I thought it was quite a milestone!

> For a 2nd language speaker making these homophonic mistakes is actually a sign of fluency. I kinda disagree because while the homophony works in (spoken) English in written it stands as a sore thumb. So yeah you will make it if you only heard it but doesn't know the written form. (And in their native language it's probably two unrelated words, so that might intensify the feeling of wrongness)

For what its worth as a non-native speaker I too started making this kind of errors when my English became fluent enough.

Re: ECC matters

#327

Post by eevilspock »

slumdev wrote: post #291
maria_weber23 wrote: post #274

Earlier quoted context omitted.

ECC memory can't eliminate the chances of these failures entirely. They can still happen. Making software resilient against bitflips in memory seems very difficult though, since it not only affects data, but also code. So in theory the behavior of software under random bit flips is well... Random. You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. I…

Error-correcting code (the "ECC" in ECC) is just a quorum at the bit level.

I'm surprised that the other replies don't grasp this. This is the proper level to do the quorum.

Doing quorum at the computer level would require synchronizing parallel computers, and unless that synchronization were to happen for each low level instruction, then it would have to be written into the software to take a vote at critical points. This is going to be greatly detrimental both to throughput and software complexity.

I guess you could implement the quorum at the CPU level... e.g. have redundant cores each with their own memory. But unless there was a need to protect against CPU cores themselves being unreliable, I don't see this making sense either.

At the end of the day, at some level, it will always come down to probabilities. "Software engineering principles" will never eliminate that.

Re: ECC matters

#328

Post by wmf »

_0ffh wrote: post #289

Please someone correct me if I'm wrong, but as far as I can remember memory with extra capacity for error detection used to be a rather common thing on early PCs. That really only changed a couple of decades in, in order to be able to offer lower prices to home users who didn't know or care about the difference. Probably about the time, or earlier, when with some hard disk manufacturers megabytes suddenly shrunk to 1…

Yes, PCs used to use parity memory.

Re: ECC matters

#329

Post by giantrobot »

maria_weber23 wrote: post #274
nostrademons wrote: post #237

I still remember Craig Silverstein being asked what his biggest mistake at Google was and him answering "Not pushing for ECC memory." Google's initial strategy (c. 2000) around this was to save a few bucks on hardware, get non-ECC memory, and then compensate for it in software. It turns out this is a terrible idea, because if you can't count on memory being robust against cosmic rays, you also can't count on the soft…

ECC memory can't eliminate the chances of these failures entirely. They can still happen. Making software resilient against bitflips in memory seems very difficult though, since it not only affects data, but also code. So in theory the behavior of software under random bit flips is well... Random. You probably would have to use multiple computers doing the same calculation and then take the answer from the quorum. I…

I don't think ECC is going to give anyone a false sense of security. The issue at Google's scale is they had to spend thousands of person-hours implementing in software what they would have gotten for "free" with ECC RAM. Lacking ECC (and generally using consumer-level hardware) compounded scale and reliability problems or at least made them more expensive than they might otherwise had been.

Using consumer hardware and making up reliability with redundancy and software was not a bad idea for early Google but it did end up with an unforeseen cost. Just a thousand machines in a cosmic ray proof bunker will end up with memory errors ECC will correct for free. It's just reducing the surface area of "potential problems".

Re: ECC matters

#330

Post by projektfu »

dijit wrote: post #25
sys_64738 wrote: post #14

ECC memory is predominantly used in servers where failure absolutely must be identified and logged. The desktop market to a lesser extent due to lack of mission critical tasks being run from there.

There are situations though, where you’re working on a document and the documents “save” format is a memory dump. Corruption for things of that type (Adobe RAW for example) would remove data. It might present itself as a 1pixel colour difference, but it could be more damaging (incorrect finances, in accounting software for example). Software trusts memory; but memory can lie. That’s dangerous.

Perhaps consumer-grade software that needs guarantees of correctness should be using error correction in software. For example, database records for financial software, DNS, e-mail addresses, etc.
Post reply on HN