Live data from Hacker News

The design of littlefs: A fail-safe filesystem designed for microcontrollers

github.com

11–20 of 74 posts

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#13

Earlier quoted context omitted.

Also why not chuck a capacitor in the mix to provide a little power buffer to shutdown the filesystem gracefully.

It adds quite some space and probably costs as much as the MCU.

not as simple either, you still need to detect that the power is gone or fading independent of the capacitor...which all adds up

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#14

"Usually, microcontroller code is simple and reactive, with no concept of a shutdown routine." Really? Nobody uses brownout interrupts to lock down on power loss? I almost always do a few things as the power is fading, flagging the situation at the very least in order to clean up when power is restored.

In one project I worked on we monitored the 24V rail and when it was dropping, the MCU had enough time to do cleanup /shutdown work until the 3.3V supply dropped.

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#15
> and the unfortunate fact is you can't traverse a tree with constant RAM

This is not true. See Knuth, The Art of Computer Programming Vol.1, § 2.3.5).

Generally I wouldn't recommend naive SDW as a good strategy for filesystems, but if you have a fixed number of processes (common in microcontroller applications) it might be worth trying.

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#16
post #5

Afaict checksums only cover metadata and it doesn't do any read integrity checking, isn't that bad when talking to raw flash devices since it relies on them accurately reporting errors?

Microcontrollers almost always use NOR flash, which is much less error prone than the NAND flash in your ssd.

Still, if you need ECC (maybe it's a medical application), the company designing the MCU would build it directly in to the embedded flash. The foundry might have a standard 128-bit word size option, and an extended 144-bit word size option, giving you 16 extra bits per word for whatever ECC or secded code you want to put in. From there, it's on you to build in the right error reporting mechanisms. It's certainly worth checking whether or not littleFS property bubbles errors up if a read operation flags the data as bad.

All this is assuming internal flash, I'm not sure whether external SPI flash offer similar extended word size options. I don't recall seeing any but maybe they are out there.

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#18
> Logging filesystem are beautifully elegant. With a checksum, we can easily detect power-loss and fall back to the previous state by ignoring failed appends.

An edge case when designing a log-structured file system is that a corrupt checksum in a log entry could actually mean one of three things: power loss while writing the last transaction, a misdirected write or bitrot, or a misdirected read.

If you simply read the log up until the corrupt checksum, assuming this can only mean one thing, then you might fall too far back and lose data that has been acknowledged to the user as synced. This could in turn then break the guarantees required for implementing things like Paxos, which could ultimately wreak havoc with a distributed system.

To detect power-loss, you need more than just a checksum. You need a checksum, but you also need a way to "snap" the new log tail onto the old log tail in a two-phase commit. So you write out all your data, then you write out your new log tail, then you sync, then you snap your new log tail on using two atomic sector writes, one of which needs to be synced.

The number of syncs are the same as before, you haven't actually added any additional syncs, but this way you ensure that your log references data which exists, and you can now tell the difference between a power failure and a corrupt log entry.

A power failure is recoverable (roll back), but a corrupt log entry is unrecoverable (assuming no log entry replicas) and means the file system is unmountable.

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#19
post #5

Afaict checksums only cover metadata and it doesn't do any read integrity checking, isn't that bad when talking to raw flash devices since it relies on them accurately reporting errors?

Microcontrollers almost always use NOR flash, which is much less error prone than the NAND flash in your ssd. Still, if you need ECC (maybe it's a medical application), the company designing the MCU would build it directly in to the embedded flash. The foundry might have a standard 128-bit word size option, and an extended 144-bit word size option, giving you 16 extra bits per word for whatever ECC or secded code you…

I was talking about checksumming, not error correction. Since the meta-data blocks are already covered by CRC32 I wonder why the data blocks aren't.

Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers

#20

"Usually, microcontroller code is simple and reactive, with no concept of a shutdown routine." Really? Nobody uses brownout interrupts to lock down on power loss? I almost always do a few things as the power is fading, flagging the situation at the very least in order to clean up when power is restored.

Nope. Because brown-out detectors on micros are meant for momentary glitches and unreliable. The detectors can preserve a brown out detected bit in their reset cause registers. However, they don't guarantee the micro's core will get the interrupt in time before losing power. And if the brown-out goes down to POR levels, you won't even get the brown out cause bit and instead POR.

The best you can do for file systems is track a "dirty flag". You keep it set unless you power down willingly yourself.

Post reply on HN