Wow, this is pretty cool to read. I'm in college for CS right now and this uses most of the stuff I've learned so far. It's neat to see all of it come together.
The design of littlefs: A fail-safe filesystem designed for microcontrollers
51–60 of 74 posts
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#52It would be nice for a thing calling itself "fail-safe" to have some amount of formal evidence of its fail-safe properties by way of both checked abstract models & verified source/binary level implementation (I also think the same thing about everything posted here related to security too). I browsed through the repo and didn't see anything other than what amounted to narrowly targeted unit & integration tests.
The test scripts are mostly in Python, so even just some Hypothesis tests (property-based testing framework for Python) would be nice to try to quasi-brute force some more empirical assurance that this filesystem behaves as claimed and expected.
That said, this is likely to come in very handy, and seems definitely worth taking a look at.
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#53> Logging filesystem are beautifully elegant. With a checksum, we can easily detect power-loss and fall back to the previous state by ignoring failed appends. An edge case when designing a log-structured file system is that a corrupt checksum in a log entry could actually mean one of three things: power loss while writing the last transaction, a misdirected write or bitrot, or a misdirected read. If you simply read t…
Isn't that essentially sqlite?
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#54Earlier quoted context omitted.
Many voltage regulators have 'power good' pins that you can tie to the MCU's reset line, though.
And if you don't have this, I posted this circuit example I just made for a delayed power turn on time above. Capacitor is about 5 cents, the transistors are about 6 cents each. If you need it you need it, but power good to the reset line is better. https://i.postimg.cc/Df9N8fM5/delayed-turn-on-circuit.png Or just put a capacitor after the reset line pullup so that it effects delayed turn on during boot.
What is the voltage drop across D1 and Q2? Is that ever an issue?
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#55I like the doubled-up metadata records. It is like the mechanism used in transactional exFAT, but covers more of the metadata and is more scalable. At first I worried about the overhead but is is nothing compared to the other overheads in filesystems unless you have a crazy number of files.
I am not sure about the system that generates "random" offsets. It seems that it is based on XOR and linear checksums and I've personally used that kind of method to hash things and had wildly imbalanced bucket use. It's a disaster if you are carving work to thousands of nodes based on the hash but might not be so bad for the intended use.
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#56> Logging filesystem are beautifully elegant. With a checksum, we can easily detect power-loss and fall back to the previous state by ignoring failed appends. An edge case when designing a log-structured file system is that a corrupt checksum in a log entry could actually mean one of three things: power loss while writing the last transaction, a misdirected write or bitrot, or a misdirected read. If you simply read t…
This is not my field of expertise, so forgive me if im mistaken, but isnt 'Ohad Rodeh B-trees'[1] a simple and more elegant solution than a journal/WAL?
As far as i know its already used in Linux Btrfs and in LMDB, and i wonder why, if they were designing this from scratch, why they didnt go for this in the first place. Familiarity perhaps?
By the way i have code that deals with the SQLite Btree directly and by reading your comment now i understand why theres a need for a two-phase commit as expressed in:
BtreeCommitPhaseOne()
BtreeCommitPhaseTwo()
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#57Earlier quoted context omitted.
Also why not chuck a capacitor in the mix to provide a little power buffer to shutdown the filesystem gracefully.
Many MCUs do not appreciate slow rise-times when using their internal regulators. Then you start oversizing your regulator or providing them replacement regulators. Which then opens more cans of worms, etc, etc.
Those microcontrollers aren't reliable anyways. If you design those in you have to accept that if they have persistent state they will eventually brick themselves. If they don't have persistent state they will hang and require user intervention on a regular basis.
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#58Earlier quoted context omitted.
I don't personally usually bother, but an EE approach would be to put a diode between the 3.3V supply and the MCU/flash power supply pins with a large bulk capacitor, and use the voltage just before the diode as your brownout detection. It can't discharge back into the power supply or other parts on the board because of the diode so the capacitor just needs to store enough charge to run for a specified time after pow…
Another reply mentioned that many MCUs don't like it when power is applied too slowly. Here is a version I just made as an example that uses two MOSFETs and a resistor divider to delay application of power to the MCU and flash memory. It relies on the N-channel MOSFET turn on voltage along with the resistor divider to set the turn on voltage, and when it reaches the threshold the P-channel MOSFET turns on. Note the r…
Most mcus are fine with power being applied slowly as long as they have a separate reset signal. It's good practice to have a separate reset signal anyway, to avoid unintended behaviour while the circuit is powering up.
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#59Earlier quoted context omitted.
I don't personally usually bother, but an EE approach would be to put a diode between the 3.3V supply and the MCU/flash power supply pins with a large bulk capacitor, and use the voltage just before the diode as your brownout detection. It can't discharge back into the power supply or other parts on the board because of the diode so the capacitor just needs to store enough charge to run for a specified time after pow…
A compact and reliable 470 uF capacitor could cost comparable to to a low-end uC.
The low end uC's use so little power that a 1uF capacitor would be plenty for a few milliseconds.
Re: The design of littlefs: A fail-safe filesystem designed for microcontrollers
#60Earlier quoted context omitted.
No SSDs here, this is raw flash. This is the wear leveling layer.
I assumed so, but then why all the comparisons to file systems that are designed to run on top of a flash translation layer? I would have expected something in big bold letters at the top saying this is designed to run without an FTL, whereas the other filesystems rely on one to wear level. For example, this is right at the top: "Writing to flash is destructive. If a filesystem repeatedly writes to the same block, ev…
The word "block" is extremely generic and the semantics and sizes of "blocks" vary widely across different storage technologies, platforms and specific parts.
It's like saying "packet" -- there are a gazillion packet types, sizes and formats, and when I use the word "packet" I don't expect everyone to assume I'm just talking about ethernet framing, even though that's a really popular packet format.