Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

blog.robertelder.org

Post reply on HN
Print view View original

61–70 of 79 posts

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#61

Post by kstenerud »

MrFoof wrote: post #49
russdill wrote: post #27

Earlier quoted context omitted.

Crashes in the middle of the night are not what worries me. Who cares. It's silent data loss that can go unnoticed for a very long time. And not just a single bit. If the flip hits file system structures or file layout you can have massive silent data loss.

Yep. It's why ZFS, BTRFS, Ceph and Gluster matter. Being able to detect that data at rest has gone wrong, and being able to reconstruct the original state is a big deal. I'd like to think that as NAND continues to scale up in capacity and lower in cost, that we'll see some real shakeup to filesystems and storage where self-healing mass storage can be genuinely commoditized -- not something that's only accessible to b…

Absolutely, but these are only half the solution. You still have to be sure that the data you're passing to the filesystem is not already corrupted in memory.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#63

Post by 1letterunixname »

hayst4ck wrote: post #2

Bit flips are totally real, at scale you will definitely see them on large queries. There was a fun talk at DEFCON on bitsquatting, the process of buying 1 bit off domain names and then accepting all incoming connections. Attacks like rowhammer similarly abuse erroneous bit flips. Supposedly microsoft can detect solar activity based on the number of windows crash logs they receive. DEFCON Talk: https://www.youtube.co…

"Supposedly" is false. The sun doesn't produce cosmic ray-power particles and soft errors aren't affected by sunspot activity. It is affected by altitude and space weather, but not solar activity.

https://en.wikipedia.org/wiki/Soft_error

I was in the audience of the talk. All devices should be required to use ECC because it's a security risk. Not as much as http:// era, but silent corruption across networks and systems is a thing.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#64

Post by colechristensen »

1letterunixname wrote: post #63
hayst4ck wrote: post #2

Bit flips are totally real, at scale you will definitely see them on large queries. There was a fun talk at DEFCON on bitsquatting, the process of buying 1 bit off domain names and then accepting all incoming connections. Attacks like rowhammer similarly abuse erroneous bit flips. Supposedly microsoft can detect solar activity based on the number of windows crash logs they receive. DEFCON Talk: https://www.youtube.co…

"Supposedly" is false. The sun doesn't produce cosmic ray-power particles and soft errors aren't affected by sunspot activity. It is affected by altitude and space weather, but not solar activity. https://en.wikipedia.org/wiki/Soft_error I was in the audience of the talk. All devices should be required to use ECC because it's a security risk. Not as much as http:// era, but silent corruption across networks and syste…

> space weather, but not solar activity

Space weather is solar activity.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#65

Post by davidcuddeback »

1letterunixname wrote: post #63
hayst4ck wrote: post #2

Bit flips are totally real, at scale you will definitely see them on large queries. There was a fun talk at DEFCON on bitsquatting, the process of buying 1 bit off domain names and then accepting all incoming connections. Attacks like rowhammer similarly abuse erroneous bit flips. Supposedly microsoft can detect solar activity based on the number of windows crash logs they receive. DEFCON Talk: https://www.youtube.co…

"Supposedly" is false. The sun doesn't produce cosmic ray-power particles and soft errors aren't affected by sunspot activity. It is affected by altitude and space weather, but not solar activity. https://en.wikipedia.org/wiki/Soft_error I was in the audience of the talk. All devices should be required to use ECC because it's a security risk. Not as much as http:// era, but silent corruption across networks and syste…

The link you provided contradicts your comment.

From the link in your comment: "The average rate of cosmic-ray soft errors is inversely proportional to sunspot activity. That is, the average number of cosmic-ray soft errors decreases during the active portion of the sunspot cycle and increases during the quiet portion."

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#66

Post by cassianoleal »

johnklos wrote: post #4

ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…

> so long as the motherboard does, too (like all ASRock that I've seen).

I built a home server last year with an ASRock X570M Pro4 [0] with a Ryzen 4750 PRO (which I had to source OEM from Aliexpress as it's not sold direct). I'm not sure what's the current situation, but the only RAM I could find for it was the Kingston Server Premier KSM32ED8 [1], and the ECC premium was not fun to pay.

[0] https://www.asrock.com/MB/AMD/X570M%20Pro4/index.asp

[1] https://www.kingston.com/en/memory/server-premier/ddr4-3200m...

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#67

Post by MrFoof »

kstenerud wrote: post #61
MrFoof wrote: post #49

Earlier quoted context omitted.

Yep. It's why ZFS, BTRFS, Ceph and Gluster matter. Being able to detect that data at rest has gone wrong, and being able to reconstruct the original state is a big deal. I'd like to think that as NAND continues to scale up in capacity and lower in cost, that we'll see some real shakeup to filesystems and storage where self-healing mass storage can be genuinely commoditized -- not something that's only accessible to b…

Absolutely, but these are only half the solution. You still have to be sure that the data you're passing to the filesystem is not already corrupted in memory.

Indeed. I have a very low power storage server with tons of ECC, and the better consumer grade NASes also tend to have it. Again, the issue here is its functionally limited to enthusiasts today because of cost and the average person being completely unaware of the impact.

My hope as we move into more advanced fabrication nodes is the increasing shift to HBM in the data center space starts to at least create an HBM option in the consumer side of things. I expect at least AMD to try that push in 2027 and beyond, and I’m sure Intel is looking hard at it too. Granted, Apple is already there with its higher end silicon.

Granted, I still expect there to be product line segmentation with ECC, as it’s a good lever to push a buyer into a higher end product. Though when it’s done on package, you at least eliminate the need for a main board to actually have the traces, and the external modules to have the extra memory. So it might be the easier route to get to more ECC in the consumer space, at least for mid-range and up personal computers.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#68

Post by muro »

wa2flq wrote: post #24
muro wrote: post #15

AFAICT, no current Mac comes with ECC - do they have the same issues? If so, one doesn't hear about them too often.

The Mac Pro has ECC and is still sold. My iMac Pro has it as well.

Ah, I see, I never considered one of them, thanks. One is discontinued, the other is expected to be replaced soon, but good to know.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#69

Post by dtgriscom »

colechristensen wrote: post #64
1letterunixname wrote: post #63

Earlier quoted context omitted.

"Supposedly" is false. The sun doesn't produce cosmic ray-power particles and soft errors aren't affected by sunspot activity. It is affected by altitude and space weather, but not solar activity. https://en.wikipedia.org/wiki/Soft_error I was in the audience of the talk. All devices should be required to use ECC because it's a security risk. Not as much as http:// era, but silent corruption across networks and syste…

> space weather, but not solar activity Space weather is solar activity.

Well, mostly...

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#70

Post by toast0 »

iopq wrote: post #51
toast0 wrote: post #48

Earlier quoted context omitted.

It's pretty easy for those to be race conditions, too. Plenty of one time crashes in a fleet of thousands of machines with ECC. ECC lets you know it's almost certainly not a memory issue. 10% more fps doesn't matter at 150 fps, but it's nice when your FPS is lower. 60 -> 66 might mean you don't dip below 60 as often. 55 -> 60.5 is pretty nice too. Maybe less of a deal if you've got VRR etc.

What game runs at only 60 FPS because of a RAM issue? I know I only have a 3600, but if a game is running at 60 FPS it's 99% because of my GPU, not the RAM. Most games still run at 90+ FPS, I would love to have ECC RAM to prevent a potential one-off crash or just to know that the RAM didn't report an error when it happened. I would pay money for this! Better yet, the 3d cache CPUs don't care about RAM speed as much,…

Play games on APUs and RAM speed will make a big difference and could get you from playable to not.
Post reply on HN