Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong? • Hacker News Board

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#71

Post by BearOso » Sun, Mar 05, 2023, 6:37 PM UTC

ilyt wrote: post #23

BearOso wrote: post #14

Earlier quoted context omitted.

DDR5 has enough ECC on chip to make errors effectively impossible. It doesn't provide error data to the CPU, though, so errors in transit can still occur. This is really unlikely, though, and anything not mission-critical will no longer need the extra ECC computation on the CPU-side. (DDR5 encapsulates the memory controller).

source on that ? Did anyone tested that ? > This is really unlikely, though, and anything not mission-critical will no longer need the extra ECC computation on the CPU-side. ECC computation is done in hardware anyway

I meant the memory controller on the CPU side won't need to implement it. Obviously, full DDR5-ECC hardware exists, but the onchip ECC as a whole makes bit flips far less likely than DDR4. There's not much of a need for the complete set on consumer hardware.

Of course this is assuming random cosmic ray bit flips, not faulty hardware. And it's speaking cost-wise from the manufacturer's perspective. I'd personally like full ECC to just be the standard.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#72

Post by T3OU-736 » Sun, Mar 05, 2023, 8:27 PM UTC

https://youtu.be/aPd8MaCyw5E ("ShmooCon 2014: You Don' Have The Evidence - Forensic Imaging Tools") was quite an eye-opening talk about common tools, like the article-mentioned `dd` (and its cousin `ddrescue`) and how they deal with I/O errors.

To be clear, I do not believe that the tools are at fault - rather, the SATA/SAS/IDE controllers have a different design goal, and software tools can only do so much.

Tools like DeepSpar (HW+SW), PC-3000 (also HW+SW) allow for a scary level of nitty-gritty access to HW, including flashing SSD/HDD controller FW in case in went pear-shaped), but for data recovery - be it in a forensic context, or in a context of retrieving important irreplaceable data, I have always had a nerd-lust for those tools. Used them at a previous job, but can't ever justify the price for personal and very infrequent use. :)

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#73

Post by bartvk » Sun, Mar 05, 2023, 8:50 PM UTC

russdill wrote: post #27

johnklos wrote: post #4

ECC is good, and I genuinely wish it were more common. Thankfully, Ryzen CPUs support ECC by default (except for pre-7000 series with integrated graphics that aren't "Pro" versions), so long as the motherboard does, too (like all ASRock that I've seen). I'm running several Ryzen servers with ECC. On the other hand, there are many, many systems out there that don't have ECC, nor do they have the option to have ECC. Wh…

Crashes in the middle of the night are not what worries me. Who cares. It's silent data loss that can go unnoticed for a very long time. And not just a single bit. If the flip hits file system structures or file layout you can have massive silent data loss.

My most precious personal data are my family pictures. I protect them with par2. These are basically checksums for your files, but they provide so much added information that you can also repair your files if they are damaged.

Once I coded a shell script that verified all my photos, but I don’t bother with that anymore. I just back everything up, and if there’s ever a problem, the parity files provide an additional safety net.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#74

Post by ta988 » Mon, Mar 06, 2023, 1:45 AM UTC

undersuit wrote: post #44

ta988 wrote: post #7

I've had a lot of really strange bugs and data loss with my current build (Ryzen with Gskill memory). After running a memtest for 24h i finally saw that two of the four ram sticks were faulty (two bit flips on each only rarely and on a specific test). The company changed them but now a year later without any issues I have another one that failed in exactly the same way. This is the last time I build a non-ECC system…

My motherboard isn't rated for more than DDR4-3200 with my old cpu, a Ryzen 7 2700. I could set my memory's XMP profile up and run at DDR4-3466 and memtest would be stable for more than 24 hours but would error before 48. I backed off, DDR-3400, DDR4-3333, DDR4-3266... finally stable in memtest for 96 hours, boot into Windows and run Prime95 Blend workload, 3266 crashes in hours. I finally find a little note in my mo…

In my case I tried at 2666 and 3200 but they were still failing exactly at the same address.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#75

Post by 1letterunixname » Mon, Mar 06, 2023, 2:51 AM UTC

colechristensen wrote: post #64

1letterunixname wrote: post #63

Earlier quoted context omitted.

"Supposedly" is false. The sun doesn't produce cosmic ray-power particles and soft errors aren't affected by sunspot activity. It is affected by altitude and space weather, but not solar activity. https://en.wikipedia.org/wiki/Soft_error I was in the audience of the talk. All devices should be required to use ECC because it's a security risk. Not as much as http:// era, but silent corruption across networks and syste…

> space weather, but not solar activity Space weather is solar activity.

Not entirely. Extrasolar activity is the most likely cause for energetic cosmic rays at the Earth's surface.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#76

Post by runnerup » Mon, Mar 06, 2023, 2:55 AM UTC

silisili wrote: post #13

BearOso wrote: post #10

Earlier quoted context omitted.

I remember reading somewhere about that talk being debunked. Maybe someone more resourceful than me can find it. It was something about being more likely to be a human typo or a config change that rolled out to a bunch of machines. The statistics didn't add up, and it wasn't plausible that bit flips caused it.

Unsure about the talk, but I read a research report from someone at I think it was Cisco who purchased a second level tld of a bitsquatted US state( think statenXX.us instead of state.XX.us ). The amount of email they received was staggering, and I'd have trouble believing that many people would make that mistake in typing.

that's more "typo-squatting" than "bit squatting" but its definitely the same broad concept.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#77

Post by semi » Mon, Mar 06, 2023, 3:24 PM UTC

bartvk wrote: post #73

russdill wrote: post #27

Earlier quoted context omitted.

Crashes in the middle of the night are not what worries me. Who cares. It's silent data loss that can go unnoticed for a very long time. And not just a single bit. If the flip hits file system structures or file layout you can have massive silent data loss.

My most precious personal data are my family pictures. I protect them with par2. These are basically checksums for your files, but they provide so much added information that you can also repair your files if they are damaged. Once I coded a shell script that verified all my photos, but I don’t bother with that anymore. I just back everything up, and if there’s ever a problem, the parity files provide an additional s…

How do you protect/ensure the integrity of your par2 files?

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#78

Post by semi » Mon, Mar 06, 2023, 3:55 PM UTC

toast0 wrote: post #70

iopq wrote: post #51

Earlier quoted context omitted.

What game runs at only 60 FPS because of a RAM issue? I know I only have a 3600, but if a game is running at 60 FPS it's 99% because of my GPU, not the RAM. Most games still run at 90+ FPS, I would love to have ECC RAM to prevent a potential one-off crash or just to know that the RAM didn't report an error when it happened. I would pay money for this! Better yet, the 3d cache CPUs don't care about RAM speed as much,…

Play games on APUs and RAM speed will make a big difference and could get you from playable to not.

Or aim for extremely high FPS (>250+) on games that are not GPU bound - counterstrike being the typical example here.

Re: Imaging a Hard Drive with Non-ECC Memory – What Could Go Wrong?

#79

Post by bartvk » Mon, Mar 06, 2023, 4:44 PM UTC

semi wrote: post #77

bartvk wrote: post #73

Earlier quoted context omitted.

My most precious personal data are my family pictures. I protect them with par2. These are basically checksums for your files, but they provide so much added information that you can also repair your files if they are damaged. Once I coded a shell script that verified all my photos, but I don’t bother with that anymore. I just back everything up, and if there’s ever a problem, the parity files provide an additional s…

How do you protect/ensure the integrity of your par2 files?

In case you’re not joking: I don’t. It’s the one and only line of defense.