Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NixOS on Btrfs+tmpfs

cnx.srht.site

Post reply on HN
Print view View original

11–20 of 55 posts

Re: NixOS on Btrfs+tmpfs

#11

Post by londons_explore »

nix23 wrote: post #9
yjftsjthsd-h wrote: post #2

Neat:) I would never use btrfs myself[0], but very happy to see people exploring all variations of these ideas. The one thing that's starting to bug me though, as I read blog posts about installing nixos: why is the install process so imperative/non-declarative? Once the system is up, the whole thing fits in configuration.nix, but to get there we still have to use masses of shell commands. Is anyone working on bridgi…

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

All these "clever" filesystems can never guarantee not to run out of space for their own metadata. That's because even to delete a file they might need more space in the journal, or to un-copy-on-write some metadata.

The mistake however is that even though it isn't practical to make theoretical guarantees that the filesystem won't end up full and broken, it is very possible to make such a thing only happen in exceeding unlikely cases. One runaway dd isn't that...

Re: NixOS on Btrfs+tmpfs

#12

Post by viraptor »

londons_explore wrote: post #10
pdenton wrote: post #8

Earlier quoted context omitted.

IIRC, noatime is useful everywhere except /var/spool or /var/mail where certain daemons may depend on access time correctness.

You can't depend on access time correctness... Because someone else can come along and grep through all your files and now they're all accessed right now.

That's the theory. In practice for example courier IMAP relied (still relies?) on at least relatime for some notification aspects.

Re: NixOS on Btrfs+tmpfs

#13

Post by amelius »

londons_explore wrote: post #10
pdenton wrote: post #8

Earlier quoted context omitted.

IIRC, noatime is useful everywhere except /var/spool or /var/mail where certain daemons may depend on access time correctness.

You can't depend on access time correctness... Because someone else can come along and grep through all your files and now they're all accessed right now.

Perhaps it just triggers some wasted CPU time, as opposed to incorrect behavior.

Re: NixOS on Btrfs+tmpfs

#14

Post by traceroute66 »

viraptor wrote: post #5

> Most subvolumes can be mounted with noatime, except for /home where I frequently need to sort files by modification time. That doesn't sound right. Noatime turns off recording of the last access time, not modification.

> Most subvolumes can be mounted with noatime

This noatime thing is an old-wive's tale that needs to die.

AFAIK, most "modern" filesystems (XFS,BTRFS etc.) all default to relatime

relatime maintains atime but without the overhead

EDIT TO ADD:

Actually,I've just done a bit of searching .... relatime has been the kernel mount default since >= 2.6.30 ! [1]

[1] https://kernelnewbies.org/Linux_2_6_30 (scroll to 1.11. Filesystems performance improvements)

Re: NixOS on Btrfs+tmpfs

#15

Post by kilburn »

traceroute66 wrote: post #14
viraptor wrote: post #5

> Most subvolumes can be mounted with noatime, except for /home where I frequently need to sort files by modification time. That doesn't sound right. Noatime turns off recording of the last access time, not modification.

> Most subvolumes can be mounted with noatime This noatime thing is an old-wive's tale that needs to die. AFAIK, most "modern" filesystems (XFS,BTRFS etc.) all default to relatime relatime maintains atime but without the overhead EDIT TO ADD: Actually,I've just done a bit of searching .... relatime has been the kernel mount default since >= 2.6.30 ! [1] [1] https://kernelnewbies.org/Linux_2_6_30 (scroll to 1.11. File…

> but without the overhead

The cost of atime is an extra write every time you read something.

Relatime changes this to one atime update per day (by default), low enough that it usually doesn't matter.

However, that update per day may have significant impact when you are using Copy-on-Write filesystems (btrfs, zfs). Each time the atime field is updated you are creating a new metadata block for that file. Old blocks can be reclaimed by the garbage collector (at an extra cost), but not if they exist in some snapshot.

All of this means that if you use btrfs/zfs and have lots of small files and take snapshots at least once per day, there's a noticeable performance difference between relative and noatime.

I've been using noatime everywhere for several years and I've never noticed any downside. This is definitely my recommended solution.

Re: NixOS on Btrfs+tmpfs

#16

Post by kilburn »

> To make use of snapshots, the backup drive gotta be Btrfs as well. The compression level was turned up to 14 this time (default was 3):

Isn't this useless? My understanding is that compression is only done at file write time. When you "btrfs send" a snapshot, the data is streamed over without recompression, so there's no point in setting up a higher compression level in the backup disk.

Re: NixOS on Btrfs+tmpfs

#18

Post by KingMachiavelli »

londons_explore wrote: post #11
nix23 wrote: post #9

Earlier quoted context omitted.

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

All these "clever" filesystems can never guarantee not to run out of space for their own metadata. That's because even to delete a file they might need more space in the journal, or to un-copy-on-write some metadata. The mistake however is that even though it isn't practical to make theoretical guarantees that the filesystem won't end up full and broken, it is very possible to make such a thing only happen in exceedi…

Why can't they? For example, Btrfs reserves some storage for it's internal use which should be more than enough to update the journal to fix a full filesystem.

Re: NixOS on Btrfs+tmpfs

#19

Post by londons_explore »

KingMachiavelli wrote: post #18
londons_explore wrote: post #11

Earlier quoted context omitted.

All these "clever" filesystems can never guarantee not to run out of space for their own metadata. That's because even to delete a file they might need more space in the journal, or to un-copy-on-write some metadata. The mistake however is that even though it isn't practical to make theoretical guarantees that the filesystem won't end up full and broken, it is very possible to make such a thing only happen in exceedi…

Why can't they? For example, Btrfs reserves some storage for it's internal use which should be more than enough to update the journal to fix a full filesystem.

Calculating exactly how much you need to reserve for the worst case is a near-impossible task.

For example, say you try to delete a file, which is part of one of multiple identical snapshots, so deleting the file doesn't free up any space, but does require extra metadata to be written (since a new directory entry will be needed that shows the file is deleted in this snapshot only).

The same operation could be done for millions of files, eating up all the reserved space. End result: full disk and unusable filesystem, even for deletes.

The alternative is not to allow file deletes to use reserved space. But now when you have a full disk, some things become 'undeletable', since the only way to free space is to delete all copies of the file, but it isn't permitted to delete any one copy of the file since the intermediate state would use more disk space.

Re: NixOS on Btrfs+tmpfs

#20

Post by Fnoord »

nix23 wrote: post #9
yjftsjthsd-h wrote: post #2

Neat:) I would never use btrfs myself[0], but very happy to see people exploring all variations of these ideas. The one thing that's starting to bug me though, as I read blog posts about installing nixos: why is the install process so imperative/non-declarative? Once the system is up, the whole thing fits in configuration.nix, but to get there we still have to use masses of shell commands. Is anyone working on bridgi…

I loose every two years (when i test it again) a volume to btrfs, last time 2 month ago, with that simple "trick": -Fill your rootpartionion as root with "dd if=/dev/urandom of=./blabla bs=3m" -rm blabla && sync (we don't want to be unfair to such a fragile system) -Reboot and end up with unbootable / It's a mess, for a filesystem i would declare it as alpha stage.

A lot of consumer grade SSDs and flash (microSD, eMMC) don't like it when they are near full. That's why you should set a reserved space and quotas. Notice your dd trick requires root which ignores the reserved space. At some point, its PEBCAK.
Post reply on HN