SDcard wear leveling and translation layers (2014)

yes, reasonably priced ones have an internal ECC engine, very cheap ones don't. The Linux NAND framwork has software ECC engines for those (see also: previous discussion on HN[1]).

> was there any ecc happening in your tests?

I deliberately turned it off. Linux has an ioctl for that. If the chip supports it, you can read/write without ECC or even directly into the OOB area where ECC data would normally be stored. The problem with the more expensive MLC NAND was that it had some smarts built in. It could discover overwrite attempts, as well as certain write patterns and apply a scrambling mask to the bits, so that the stored bits would (hopefully) not diffuse over to their neighbors.

> if you started with a page with valid ecc data and then tried to bash an arbitrary payload 1 bit in it to 0, you'd have a real challenge updating the ecc data to be consistent with the updated payload

If you are implementing a raw flash driver, there is a "nandbiterrs" test that does roughly what you describe: it bashes an increasing number of bit errors into a page and reads it back with ECC re-enabled, checking how many errors are successfully corrected during read, before the ECC engine gives up.

[1] https://news.ycombinator.com/item?id=38361139

The fact that they're used in dash cams, exposed to high temperature thermal cycles, and recording tens of terabytes of data which can be more than the typical consumer HDD/SSD in it's lifetime is quite impressive.

> You cannot do single byte writes on NAND.

This isn't true - that the OP wrote here certainly does work (at least for many places I've tested and used it). I've used it to do all sorts of low level logging in small devices (and I've thoroughly tested many, many chips to complete destruction to see how things degrade). I have not yet seen a place it fails and I've done it on dozens or projects. I guess there may be places wear leveling fights with this, but if you dig into the particular device you have you can usually find low level details to make it work. The places I've pushed it are all commercial projects for which we characterize long term failure by running the flash parts fast enough to destroy the flash, to ensure that once in the field the expected lifetimes meet any guarantees we need to provide.

Also, if you've never ran a flash chip to death, try it - it's interesting. Get a small SD card and hammer it withRead/write cycles until you start getting errors, then log and watch how those errors propagate. Good stuff :)

Every chip I've tested does indeed let you erase a block (which sets all bits to 0 or 1 depending on chip), then append bits to that block by changing the bits you need.

To append, you don't need to update some other data structure, just append a single bit of the correct type, then a bit scan of the page tells you where the data ends. Or, if you have another page telling how many "slots" of yourData are used, again append s single bit. Then small appends only cost a bit.

On top of this implement wear leveling (unless the chip does - most bigger ones do, but some low level devices don't).

Other projects I've worked on professionally are products for desoldering flash from captured and destroyed enemy devices which are then read raw and reconstructed, fault tolerant and error shielding for NASA space ops with flash, reverse engineering proprietary flash protocols for Secret Service (weird, right? but they pay) uses. I somewhat often get called for consulting on precisely these types of issues. So I do know a bit about low level flash abuse :)