Btrfs Allocator Hints

forza_user wrote: post #3

I was surprised of the new attempt for performance profiles/device roles/hints when we already have a very good patch set maintained by kakra. - https://github.com/kakra/linux/pull/36 - https://wiki.tnonline.net/w/Btrfs/Allocator_Hints What do you think?

> One of the reasons why these patches are not included in the kernel is that the free space calculations do not work properly.

It seems these patches possibly fix that.

Re: Btrfs Allocator Hints

#5

Post by bjoli » Fri, Jul 11, 2025, 9:51 AM UTC

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

Re: Btrfs Allocator Hints

#6

Post by bionade24 » Fri, Jul 11, 2025, 10:49 AM UTC

bjoli wrote: post #5

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

Most likely yes, but the also envisioned periodically repacking oft multiple small data extents into one big that gets written to the HDD would wake up the HDD. And if you'd make the SSD "metadata only", browser cache and logging will keep the HDD spinning.

This feature is for performance, not the case you described.

Re: Btrfs Allocator Hints

#7

Post by the8472 » Fri, Jul 11, 2025, 11:14 AM UTC

bjoli wrote: post #5

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

I think you'd rather want a cache device (or some more complicated storage tiering) for that so that both metadata and frequently accessed files get moved to that dynamically based on access patterns. Afaik btrfs doesn't support that. LVM, bcache, device mapper, bcachefs and zfs support that (though zfs would require separate caches for reading and synchronous write). And idk which of these let you control the writeback interval.

Re: Btrfs Allocator Hints

#8

Post by ajross » Fri, Jul 11, 2025, 11:58 AM UTC

bjoli wrote: post #5

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

Just buy more RAM and you get that for free. Really I guess that's my sense of patches like this in general: while sure, filesystem research has a long and storied history and it's a very hard problem in general that attracts some of the smartest people in the field to do genius-tier work...

Does it really matter in the modern world where a vanilla two-socket rack unit has a terabyte of DRAM? Everything at scale happens in RAM these days. Everything. Replicating across datacenters gets you all the reliability you need, with none of the fussing about storage latency and block device I/O strategy.

Re: Btrfs Allocator Hints

#9

Post by homebrewer » Fri, Jul 11, 2025, 12:26 PM UTC

ajross wrote: post #8

bjoli wrote: post #5

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

Just buy more RAM and you get that for free. Really I guess that's my sense of patches like this in general: while sure, filesystem research has a long and storied history and it's a very hard problem in general that attracts some of the smartest people in the field to do genius-tier work... Does it really matter in the modern world where a vanilla two-socket rack unit has a terabyte of DRAM? Everything at scale happ…

> Does it really matter in the modern world

Considering that multiple ZFS developers get paid to make ZFS work well on petabyte-sized disk arrays with SSD caching, and one of them often reports on progress in this area in his podcasts (2.5admins.com and bsdnow if you're interested) .. then yes?

Re: Btrfs Allocator Hints

#10

Post by guenthert » Fri, Jul 11, 2025, 1:37 PM UTC

ajross wrote: post #8

bjoli wrote: post #5

I wonder if I can use a smaller SSD for this and make it avoid HDD wakeups due to some process reading metadata. That alone would make me love this feature.

Just buy more RAM and you get that for free. Really I guess that's my sense of patches like this in general: while sure, filesystem research has a long and storied history and it's a very hard problem in general that attracts some of the smartest people in the field to do genius-tier work... Does it really matter in the modern world where a vanilla two-socket rack unit has a terabyte of DRAM? Everything at scale happ…

Some time ago (back when we were using spinning rust) I was wondering whether one could bypass the latency of disk access when replicating to multiple hosts. I mean, how likely is it, that two hosts crash at the same time? Well, it turns out that there are some causes which take out multiple hosts simultaneously (a way too common occurrence seems to be diesel generators which fail to start after power failure). I think the good fellas at Amazon, Meta and Google even have stories to tell about a whole data center failing. So you need replication across data centers, but then network latency bites ya. Current NVMe storage devices are then faster (and for some access patterns nearly as fast as RAM).

And that's just at the largest scale. I'm pretty sure banks still insist that the data is written to (multiple) disks (aka "stable storage") before completing a transaction.