311–320 of 331 posts
Earlier quoted context omitted.
Who would that help? MacOS also defaults to a non-portable FS for likely similar reasons, if one was being cynical.
It would help users using USB sticks, external drives? Couple it with encrypted zfs send/receive for cross platform secure backups.
Earlier quoted context omitted.
As I understand, L2ARC doesn't work across reboots which unfortunately makes it almost useless for systems that get rebooted regularly, like desktops.
L2ARC has had persistence support for a few years now.
Earlier quoted context omitted.
One reason why it might be a good idea to use higher quality drives when using ZFS is because it seems like in some scenarios ZFS can result in more writes being done to the drive than when other file systems are used. This can be a problem for some QLC and TLC drives that have low endurance. I'm in the process of setting up a server at home and was testing a few different file systems. I was doing a test where I had…
One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. The default is 5 seconds, but I usually increase mine to 20. When writing a lot of data, it'll get flushed to disk more often, so this timer is only for when you're writing small amounts of data like the test you just described. > like might happen for so…
My mind jumped at that too when I first read parent's comment. But presumably he's writing other files to disk too. Not just that one file. :)
Earlier quoted context omitted.
Heavy synchronous IO from incredibly frequent fsync is a weak point. You can make it better using SLOG devices. I realize what I am about to say is not what you want to hear, but any application doing excessive fsync operations is probably doing things wrong. This is a view that you will find prevalent among all filesystem developers (i.e. the ext4 and XFS guys will have this view too). That is because all filesystem…
Actually this was discussed recently [0]. While everybody knows it's not efficient, it's required to keep update process resilient against unwanted shutdowns (like power losses which corrupt the filesystem due to uncommitted work left on the filesystem). > From a power loss perspective, if power is lost when installing a package into the filesystem, you need to repair the package. Yes, but at least you have all the f…
https://lists.debian.org/debian-devel/2024/12/msg00540.html
It claims that the fsync is needed to avoid the file appearing at the final location with a zero length after a power loss. This is not true on ZFS.
ZFS puts every filesystem operation into a transaction group that is committed atomically about every 5 seconds by default. On power loss, the transaction group either succeeds or never happens. The result is that even without using fsync, there will never be a zero length file at the final location because the rename being part of a successful transaction group commit implies that the earlier writes also were part of a successful transaction group commit.
The result is that you can use --force-unsafe-io with dpkg on ZFS, things will run faster and there should be no issues for power loss recovery as far as zero length files go.
The following email mentions that sync() had been used at one point but caused problems when flash drives were connected, so it was dropped:
https://lists.debian.org/debian-devel/2024/12/msg00597.html
The timeline is unclear, but I suspect this happened before Linux 2.6.29 introduced syncfs(), which would have addressed that. Unfortunately, it would have had problems for systems with things like a separate /usr mount, which requires the package manager to realize multiple syncfs calls are needed. It sounds like dpkg was calling sync() per file, which is even worse than calling fsync() per file, although it would have ensured that the directory entries for prior files were there following a power loss event.
The email also mentions that fsync is not called on directories. The result is that a power loss event (on any Linux filesystem, not just ZFS) could have the files missing from multiple packages marked as installed in the package database, which is said to use fsync to properly record installations. I find this situation weird since I would use sync() to avoid this, but if they are comfortable having systems have multiple “installed” packages missing files in the filesystem after a power loss, then there is no need to use sync().
Earlier quoted context omitted.
Hi! I am quite a beginner when it comes to file systems. Would this sync effect not be helped by direct IO in ZFS's case? Also, given that you seem quite knowledgeable of the topic, what is your go-to backup solution? I initially thought about storing `zfs send` files into backblaze (as backup at a different location), but without recv-ing these, I don't think the usual checksumming works properly. I can checksum the…
No, it will not. It would be helped by APT switching to using a single sync/syncfs call after installing all files, which is the performant way to do what it wants on Linux: https://www.man7.org/linux/man-pages/man2/sync.2.html
DPKG will write each file to a temporary location and then rename it to the final location. On ext4, without fsync, when a power loss event occurs, it is possible for the rename to the final location to be done, without any of the writes such that you have a zero length file. On ZFS, the rename being done after the writes means that the rename being done implies the writes were done due to the sequential nature of ZFS’ transaction group commit, so the file will never appear in the final location without the file contents following a power loss event, which is why ZFS does not need the fsync there.
After years in the making ZFS raidz expansaion is finally here. Major features added in release: - RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime. - Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality. - Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where cachi…
So if I’m running a Proxmox on ZFS and NVMEs, will I be better off enabling Direct IO when 2.3 gets rolled out? What are the use cases for it?
Earlier quoted context omitted.
I'm not yet familiar with zfs and couldn't find it in the release note: Does expansion only works with disk of the same size? Or is adding are bigger/smaller disks possible or do all disk need to have the same size?
You can use different sized disks, but RAID-Z will truncate the space it uses to the lowest common denominator. If you increase the lowest common denominator, RAID-Z should auto-expand to use the additional space. All parity RAID technologies truncate members to the lowest common denominator, rather than just ZFS.
Earlier quoted context omitted.
One reason why it might be a good idea to use higher quality drives when using ZFS is because it seems like in some scenarios ZFS can result in more writes being done to the drive than when other file systems are used. This can be a problem for some QLC and TLC drives that have low endurance. I'm in the process of setting up a server at home and was testing a few different file systems. I was doing a test where I had…
One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. The default is 5 seconds, but I usually increase mine to 20. When writing a lot of data, it'll get flushed to disk more often, so this timer is only for when you're writing small amounts of data like the test you just described. > like might happen for so…
I don't believe that zfs_txg_timeout setting would make much of a difference for the test I described where I was doing synchronous writes.
> On Linux, I think journald would be aggregating your logs from multiple services so at least you wouldn't be incurring that cost on a per-program basis.
The server I'm setting up will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps. Some of the logging could be aggregated but there will be multiple types of I/O (various types of databases, app updates, file server, etc...) and I wanted to get an idea of how much file system overhead there might be in a worst case kind of scenario.
> I sure hope I've upgraded SSDs by the year 2065.
Since I'll be running a lot of stuff on the server, I'll probably have quite a bit more writing going on than the test I described so if I used ZFS I believe the SSD could reach its rated endurance in just several years.
Earlier quoted context omitted.
One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. The default is 5 seconds, but I usually increase mine to 20. When writing a lot of data, it'll get flushed to disk more often, so this timer is only for when you're writing small amounts of data like the test you just described. > like might happen for so…
>I sure hope I've upgraded SSDs by the year 2065. My mind jumped at that too when I first read parent's comment. But presumably he's writing other files to disk too. Not just that one file. :)
Yes, there will be much more going on than the simple test I was doing. The server will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps.
After years in the making ZFS raidz expansaion is finally here. Major features added in release: - RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime. - Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality. - Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where cachi…
So if I’m running a Proxmox on ZFS and NVMEs, will I be better off enabling Direct IO when 2.3 gets rolled out? What are the use cases for it?