Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NixOS on Btrfs+tmpfs

cnx.srht.site

Post reply on HN
Print view View original

21–30 of 55 posts

Re: NixOS on Btrfs+tmpfs

#21

Post by curt15 »

nix23 wrote: post #9
yjftsjthsd-h wrote: post #2

Neat:) I would never use btrfs myself[0], but very happy to see people exploring all variations of these ideas. The one thing that's starting to bug me though, as I read blog posts about installing nixos: why is the install process so imperative/non-declarative? Once the system is up, the whole thing fits in configuration.nix, but to get there we still have to use masses of shell commands. Is anyone working on bridgi…

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

How does zfs handle that test?

Re: NixOS on Btrfs+tmpfs

#22

Post by jdc »

traceroute66 wrote: post #14
viraptor wrote: post #5

> Most subvolumes can be mounted with noatime, except for /home where I frequently need to sort files by modification time. That doesn't sound right. Noatime turns off recording of the last access time, not modification.

> Most subvolumes can be mounted with noatime This noatime thing is an old-wive's tale that needs to die. AFAIK, most "modern" filesystems (XFS,BTRFS etc.) all default to relatime relatime maintains atime but without the overhead EDIT TO ADD: Actually,I've just done a bit of searching .... relatime has been the kernel mount default since >= 2.6.30 ! [1] [1] https://kernelnewbies.org/Linux_2_6_30 (scroll to 1.11. File…

I prefer lazyatime.

Re: NixOS on Btrfs+tmpfs

#23

Post by vladvasiliu »

Fnoord wrote: post #20
nix23 wrote: post #9

Earlier quoted context omitted.

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

A lot of consumer grade SSDs and flash (microSD, eMMC) don't like it when they are near full. That's why you should set a reserved space and quotas. Notice your dd trick requires root which ignores the reserved space. At some point, its PEBCAK.

> A lot of consumer grade SSDs and flash (microSD, eMMC) don't like it when they are near full

I've heard about this, but my understanding was that when this happens, performance becomes extremely poor. While that may be quite bad, it's still worlds apart from losing data.

There's also the fact that the user may have partitioned the drive in a such a way to prevent it from ever filling up. Even root can't fill the partition beyond its size. Here, you have to go out of your way to make sure the partition doesn't fill out, or else you have a bad time. Shit happens, so this does look like a FS bug to me, much more than PEBCAK.

Re: NixOS on Btrfs+tmpfs

#24

Post by nix23 »

curt15 wrote: post #21
nix23 wrote: post #9

Earlier quoted context omitted.

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

How does zfs handle that test?

dd says "no space left" and kills itself, "df" says 100% full, "zpool list" says 98% 80GB free...it's like black magic that ZFS knows to keep some space for itself ;)

And to other comments:

It was a DC-Harddisk, and NO, not even root should be capable destroying the filesystem by simply write to it, it's not 1970 anymore.

Calculating the "to reserve" Metadata-block should be rather trivial since it's ONE big file. And it's not dd that is the problem, it's btrfs that cannot handle a process that writes ONE BIG file.

Re: NixOS on Btrfs+tmpfs

#25

Post by vladvasiliu »

curt15 wrote: post #21
nix23 wrote: post #9

Earlier quoted context omitted.

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

How does zfs handle that test?

I haven't tried that particular test, but I've had a ZFS drive filled to the brim. I just deleted a bunch of files and was back to normal. This was on a computer with a single pool, also used for booting. The system didn't even crash or anything.

Re: NixOS on Btrfs+tmpfs

#26

Post by grumpyprole »

kaba0 wrote: post #7
genghizkhan wrote: post #6

I would prefer to do this on zfs, for which there is a lovely installation guide on the openzfs docs site. https://openzfs.github.io/openzfs-docs/Getting%20Started/Nix...

I can vouch for how good it works. Using it on my personal laptop.

I tried ZFS-on-linux with Ubuntu 21.10 and it ate my data (ZFS panics when accessing certain files). Sure, Ubuntu does have a habit of using unstable kernels, but I was still disappointed. It should be stable at this point.

Re: NixOS on Btrfs+tmpfs

#27

Post by xorcist »

kilburn wrote: post #15
traceroute66 wrote: post #14

Earlier quoted context omitted.

> Most subvolumes can be mounted with noatime This noatime thing is an old-wive's tale that needs to die. AFAIK, most "modern" filesystems (XFS,BTRFS etc.) all default to relatime relatime maintains atime but without the overhead EDIT TO ADD: Actually,I've just done a bit of searching .... relatime has been the kernel mount default since >= 2.6.30 ! [1] [1] https://kernelnewbies.org/Linux_2_6_30 (scroll to 1.11. File…

> but without the overhead The cost of atime is an extra write every time you read something. Relatime changes this to one atime update per day (by default), low enough that it usually doesn't matter. However, that update per day may have significant impact when you are using Copy-on-Write filesystems (btrfs, zfs). Each time the atime field is updated you are creating a new metadata block for that file. Old blocks ca…

If you don't use atime, there's no need to keep it around. Daily or otherwise. Unless you have reason to believe an application uses atime, it probably doesn't.

I've used noatime by default, except for few cases where I know it is used, in professional settings for probably two decades. Hopefully you know what kind of appliciation you are running. There are many parameters in a system and this is just one of them.

The only times I've seen atime used has been for a two queues, and only in the case of "has this file changes since last it was read". And that is precisely what relatime is for, the daily update is just an optional extra.

Re: NixOS on Btrfs+tmpfs

#28

Post by nix23 »

londons_explore wrote: post #11
nix23 wrote: post #9

Earlier quoted context omitted.

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

All these "clever" filesystems can never guarantee not to run out of space for their own metadata. That's because even to delete a file they might need more space in the journal, or to un-copy-on-write some metadata. The mistake however is that even though it isn't practical to make theoretical guarantees that the filesystem won't end up full and broken, it is very possible to make such a thing only happen in exceedi…

>it is very possible to make such a thing only happen in exceeding unlikely cases. One runaway dd isn't that...

It's not dd, it's one process run by root who fills the filesystem with one big file. That's like the first thing i would test if it can destroy my filesystem.

It's really the filesystem responsibility, if it needs to reserve 30% so be it, if it need's more because i wrote billions of files so be it, (even if it says "sorry i told you i have 50GB free but because you wrote so many small files it's now just 45GB" after all they just can make a estimation) so be it. But it's the filesystem job to tell me how much ~free space that i have, and stop writing before it really/internally cant take anymore. And NOT to kill itself because i alloc 100% of it, there's is just no excuse. That's the filesystem's responsibility.

PS: The clever ZFS survives that "unlikely" test easily.

Re: NixOS on Btrfs+tmpfs

#29

Post by nix23 »

vladvasiliu wrote: post #23
Fnoord wrote: post #20

Earlier quoted context omitted.

A lot of consumer grade SSDs and flash (microSD, eMMC) don't like it when they are near full. That's why you should set a reserved space and quotas. Notice your dd trick requires root which ignores the reserved space. At some point, its PEBCAK.

> A lot of consumer grade SSDs and flash (microSD, eMMC) don't like it when they are near full I've heard about this, but my understanding was that when this happens, performance becomes extremely poor. While that may be quite bad, it's still worlds apart from losing data. There's also the fact that the user may have partitioned the drive in a such a way to prevent it from ever filling up. Even root can't fill the pa…

>Shit happens, so this does look like a FS bug to me, much more than PEBCAK.

Exactly, a swimming-pool should never explode if you overfill it, however it's the users responsibility to turn the water off to prevent "data/water-loss"

That's why we made filesystems, preserve and organize data and tell the user/system when it cant take anymore.

Re: NixOS on Btrfs+tmpfs

#30

Post by genghizkhan »

grumpyprole wrote: post #26
kaba0 wrote: post #7

Earlier quoted context omitted.

I can vouch for how good it works. Using it on my personal laptop.

I tried ZFS-on-linux with Ubuntu 21.10 and it ate my data (ZFS panics when accessing certain files). Sure, Ubuntu does have a habit of using unstable kernels, but I was still disappointed. It should be stable at this point.

I've never tried Ubuntu, but zfs has been rock solid on Fedora, Arch and Debian for me. Whatever the issue that hit you was, I hope recent versions of Ubuntu/zfs have fixed it.
Post reply on HN