Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unpowered SSDs slowly lose data

xda-developers.com

Post reply on HN
Print view View original

241–250 of 341 posts

Re: Unpowered SSDs slowly lose data

#241

Post by johncolanduoni »

lxgr wrote: post #214
DeepSeaTortoise wrote: post #209

Earlier quoted context omitted.

IMO it's exactly the right layer, just like for ECC memory. There's a lot of potential for errors when the storage controller processes and turns the data into analog magic to transmit it. In practice, this is a solved problem, but only until someone makes a mistake, then there will be a lot of trouble debugging it between the manufacturer certainly denying their mistake and people getting caught up on the usual susp…

> There's a lot of potential for errors when the storage controller processes and turns the data into analog magic to transmit it. That's a physical layer, and as such should obviously have end-to-end ECC appropriate to the task. But the error distribution shape is probably very different from that of bytes in NAND data at rest, which is different from that of DRAM and PCI again. For the same reason, IP does not do e…

> And sticking with that example, once it becomes TCP's job to handle packet loss due to transmission errors (instead of just congestion), things go south pretty quickly.

Outside of wireless links (where FEC of some degree is necessary regardless) this is mostly because TCP’s checksum is so weak. QUIC for example handles this much better, since the packet’s authenticated encryption doubles as a robust error detecting code. And unlike TLS over TCP, the connection is resilient to these failures: a TCP packet that is corrupted but passes the TCP checksum will kill the TLS connection on top of it instead of retransmitting.

Re: Unpowered SSDs slowly lose data

#243

Post by mbreese »

dotancohen wrote: post #223
mbreese wrote: post #205

Earlier quoted context omitted.

1) you have to have an OS that supports it. 2) even if your OS supports it, you may have difficulty using it for your root volume, so partitioning is probably required. 2a) in your case you may not want to use it on your boot volume which would negate the SSD benefit for you. 3) it is recommended that you have ECC RAM due to the checksums. This isn’t a hard and fast requirement, but it does make you more resilient to…

I use Debian at home, with separate boot, /, and /home/ partitions. I have no idea what type of cheap memory is stuffed into the motherboard - it's certainly not homogeneous. I do prioritise resiliency over speed, or even space. Still something I should look into? Thank you!

The servers I use ZFS on are Debian, so it’s well supported in that way. I’m pretty sure ZFS on Debian uses dkms, so if you want to try it on a data partition, it will work.

Still, unless you want to tinker with something new I can’t really recommend it. Would it work? Yes. Do you need it? No. You’re probably fine with whatever FS you currently have running. ZFS works on Debian, but it’s not first-party support (due to licensing). Do I think you’d have issues if you wanted to try it? Probably not. I’m just conservative in what I’d recommended for a daily use machine. I prioritize working over everything else, so I’d hate for you to try it and end up with a non working system.

Here’s what I’d recommend instead - try it in a VM first. See if you can get it to work in a system setup like yours. See if it’s something that you like. If you want to use it on your primary machine, then you’ll be able to make a more informed decision.

Re: Unpowered SSDs slowly lose data

#244

Post by mghackerlady »

This is a really good case for better file systems with built in error correction and self healing. On linux they have btrfs which kinda does this, and some support for zfs. In the BSD land we have zfs and hammer2. Does NT or Mac have anything like this? I think Mac might have some unofficial zfs support but I don't know the state that's in

Re: Unpowered SSDs slowly lose data

#245

Post by johnebgd »

mghackerlady wrote: post #244

This is a really good case for better file systems with built in error correction and self healing. On linux they have btrfs which kinda does this, and some support for zfs. In the BSD land we have zfs and hammer2. Does NT or Mac have anything like this? I think Mac might have some unofficial zfs support but I don't know the state that's in

The self healing only works if you have redundant drives. If all SSD’s are like this maybe they need alarms that sound as an onboard battery gets low to ensure someone plugs them in.

Re: Unpowered SSDs slowly lose data

#246

Post by [deleted] »

anthk wrote: post #207
ACCount37 wrote: post #162

Earlier quoted context omitted.

Because no one is willing to pay for SLC. Those QLC NAND chips? Pretty much all of them have an "SLC mode", which treats each cell as 1 bit, and increases both write speeds and reliability massively. But who wants to have 4 times less capacity for the same price?

Myself wants. I remember when the UBIFS module (or some kernel settings) for the Debian kernel was MLC against SLC. You could store 4X more data now, but at a cost of really bad reability: A SINGLE bad shutdown and your partitions would be corrupted up to the point of not being able to properly boot any more, having to reflash the NAND.

[deleted]

Re: Unpowered SSDs slowly lose data

#247

Post by moffkalast »

anthk wrote: post #207
ACCount37 wrote: post #162

Earlier quoted context omitted.

Because no one is willing to pay for SLC. Those QLC NAND chips? Pretty much all of them have an "SLC mode", which treats each cell as 1 bit, and increases both write speeds and reliability massively. But who wants to have 4 times less capacity for the same price?

Myself wants. I remember when the UBIFS module (or some kernel settings) for the Debian kernel was MLC against SLC. You could store 4X more data now, but at a cost of really bad reability: A SINGLE bad shutdown and your partitions would be corrupted up to the point of not being able to properly boot any more, having to reflash the NAND.

Well then buy an industrial SSD, they're something like 80-240 GB and you get power loss protection capacitors too. Just not the datacenter ones, those melt immediately without rack airflow.

Re: Unpowered SSDs slowly lose data

#248

Post by chungy »

dotancohen wrote: post #223
mbreese wrote: post #205

Earlier quoted context omitted.

1) you have to have an OS that supports it. 2) even if your OS supports it, you may have difficulty using it for your root volume, so partitioning is probably required. 2a) in your case you may not want to use it on your boot volume which would negate the SSD benefit for you. 3) it is recommended that you have ECC RAM due to the checksums. This isn’t a hard and fast requirement, but it does make you more resilient to…

I use Debian at home, with separate boot, /, and /home/ partitions. I have no idea what type of cheap memory is stuffed into the motherboard - it's certainly not homogeneous. I do prioritise resiliency over speed, or even space. Still something I should look into? Thank you!

I use ZFS on both my desktop and laptop each with Linux (in addition to a server, also running ZFS, but on FreeBSD). It's actually really not terribly hard, but I might be biased since I've been doing since it 2011 :)

If you can/are willing to use UEFI, ZFSBootMenu is a Linux oriented solution that replicates the power of FreeBSD's bootloader, so you can manage snapshots and boot environments and rollback checkpoints all at boot without having to use recovery media (that used to be required when doing ZFS on Linux). Definitely worth looking into: https://zfsbootmenu.org/

Re: Unpowered SSDs slowly lose data

#249

Post by Aurornis »

userbinator wrote: post #168
ACCount37 wrote: post #162

Earlier quoted context omitted.

Because no one is willing to pay for SLC. Those QLC NAND chips? Pretty much all of them have an "SLC mode", which treats each cell as 1 bit, and increases both write speeds and reliability massively. But who wants to have 4 times less capacity for the same price?

4 times less capacity but 100x or more endurance or retention at the same price looks like a great deal to me. Alternatively: do you want to have 4x more capacity at 1/100th the reliability? Plenty of people would be willing to pay for SLC mode. There is an unofficial firmware hack that enables it: https://news.ycombinator.com/item?id=40405578 1TB QLC SSDs are But why won't the manufacturers let you choose? The real…

> Alternatively: do you want to have 4x more capacity at 1/100th the reliability?

If the original drive has sufficient reliability, then yes I do want that.

And the majority of consumers do, too.

Chasing absolute extreme highest powered off durability is not a priority for 99% of people when the drives work properly for typical use cases. I have 5 year old SSDs where the wear data is still in the single digit percentages despite what I consider moderately heavy use.

> I have an old SLC USB drive which is only 512MB, but it's nearly 20 years old and some of the very first files I wrote to it are still intact (I last checked several months ago, and don't expect it's changed since then.) It has probably had a few hundred full-drive-writes over the years --- well worn-out by modern QLC/TLC standards, but barely-broken-in for SLC.

Barely broken in, but also only 512MB, very slow, and virtually useless by modern standards. The only positive is that the files are still intact on that old drive you dusted off.

This is why the market doesn’t care and why manufacturers are shipping TLC and QLC: They aren’t doing a planned obsolescence conspiracy. They know that 20 years from now or even 10 years from now that drive is going to be so outdated that you can get a faster, bigger new one for pocket change.

Re: Unpowered SSDs slowly lose data

#250

Post by lxgr »

johncolanduoni wrote: post #241
lxgr wrote: post #214

Earlier quoted context omitted.

> There's a lot of potential for errors when the storage controller processes and turns the data into analog magic to transmit it. That's a physical layer, and as such should obviously have end-to-end ECC appropriate to the task. But the error distribution shape is probably very different from that of bytes in NAND data at rest, which is different from that of DRAM and PCI again. For the same reason, IP does not do e…

> And sticking with that example, once it becomes TCP's job to handle packet loss due to transmission errors (instead of just congestion), things go south pretty quickly. Outside of wireless links (where FEC of some degree is necessary regardless) this is mostly because TCP’s checksum is so weak. QUIC for example handles this much better, since the packet’s authenticated encryption doubles as a robust error detecting…

Ah, I meant go south in terms of performance, not correctness. Most TCP congestion control algorithms interpret loss exclusively as a congestion signal, since that's what most lower layers have historically presented to it.

This is why newer TCP variants that use different congestion signals can deal with networks that violate that assumption better, such as e.g. Starlink: https://blog.apnic.net/2024/05/17/a-transport-protocols-view...

Other than that, I didn't realize that TLS has no way of just retransmitting broken data without breaking the entire connection (and a potentially expensive request or response with it)! Makes sense at that layer, but I never thought about it in detail. Good to know, thank you.

Post reply on HN