Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs Coming to Fedora 33

fedoramagazine.org

Post reply on HN
Print view View original

131–140 of 140 posts

Re: Btrfs Coming to Fedora 33

#131

Post by throw0101a »

tinus_hn wrote: post #129
throw0101a wrote: post #127

Earlier quoted context omitted.

> Clearly you weren’t around in the days when NFS was a thing. I was and am. I currently help admin an HPC environment with several petabytes of NFS storage. And the main problems I've experienced with NFS have generally been with Linux locking. I've had a lot less drama with Solaris and BSD NFS (though I did uncover two Solaris NFS server bugs over the years). Whenever I have NFS issues I generally start with the as…

If you think applications work on NFS by just expecting general Unix filesystem semantics you’re in for a surprise. The fact you’re not having trouble just means either the application writers put in the effort to properly support it, or your use case is simple enough not to run into problems.

> either the application writers put in the effort to properly support it

Given the hiccups that we've had with MySQL and locking, I'm not sure that's the case. (Less so with Postgres.)

I have only a moderate amount of knowledge of the applications that the dozen research groups use on the cluster, I just make sure the bits flow and haven't heard too many complaints over the years.

Re: Btrfs Coming to Fedora 33

#132

Post by kim0 »

wtallis wrote: post #44
schoolornot wrote: post #40

Earlier quoted context omitted.

Can you explain the reasoning for this?

On a copy-on-write filesystem, modification of part of a file means fragmenting it, which carries a large performance penalty. Disabling the CoW behavior means you either cannot take snapshots of the file, or that snapshots need to duplicate the file in its entirety, but it allows for quick modification of the file in-place without fragmenting.

Is that assuming HDDs not SSDs ?

Re: Btrfs Coming to Fedora 33

#133

Post by tinus_hn »

throw0101a wrote: post #131
tinus_hn wrote: post #129

Earlier quoted context omitted.

If you think applications work on NFS by just expecting general Unix filesystem semantics you’re in for a surprise. The fact you’re not having trouble just means either the application writers put in the effort to properly support it, or your use case is simple enough not to run into problems.

> either the application writers put in the effort to properly support it Given the hiccups that we've had with MySQL and locking, I'm not sure that's the case. (Less so with Postgres.) I have only a moderate amount of knowledge of the applications that the dozen research groups use on the cluster, I just make sure the bits flow and haven't heard too many complaints over the years.

Clearly packages like MySQL and PostGreSQL made sure they work on NFS or they would very clearly tell you not to use it because they’d know it’s a recipe for disaster. The issues I am referring to typically are with things like homegrown shell scripts or research software.

Re: Btrfs Coming to Fedora 33

#134

Post by codys »

cmurf wrote: post #117
codys wrote: post #76

Earlier quoted context omitted.

My experience with btrfs is from a few years ago, so it's possible they've changed the semantics here since then, but this might be applicable: Btrfs requires a regularly run administrative task via "btrfs balance". Balancing consolidates data between partially filled block groups, which will restore consumed space even if a block group is not entirely empty. Typically, one runs balance with a set of gradually increa…

What is the source of "Btrfs requires a regularly run administrative task...". No file system should require this kind of regular hand holding. If it does, it's a bug, and needs to be reported and fixed, not papered over with user initiated maintenance tasks. It is acceptable to have utilities for optimizing the layout from time to time, but they should be considered optimizations, not requirements. If it becomes req…

There are whole repos[1] dedicated to scripts to perform the delicate work of running `btrfs balance` in various ways to ensure space is recovered.

openSuse (a distro which notably defaults to btrfs) packages these btrfsmaintenance scripts [2], and it appears they may have included it in their default install (I can't find a list of packages). Their wiki page on disabling btrfsmaintenance [5] implies that if disabled, manual maintenance (presumably via running, among other commands, some form of balance) is needed.

There's also an entry in the btrfs wiki [3] that indicates running `btrfs balance` will recover unused space in some cases. It helpfully notes that prior to "at least 3.14", balance was "sometimes" needed to recover free space in a file-system full state. (The lack of precision here doesn't inspire confidence)

Another btrfs wiki page [4] indicates that running balance may be needed to recover space "after removing lots of files or deleting snapshots".

1: https://github.com/kdave/btrfsmaintenance 2: https://software.opensuse.org/package/btrfsmaintenance 3: https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22bal... 4: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balanc... 5: https://en.opensuse.org/SDB:Disable_btrfsmaintenance

Re: Btrfs Coming to Fedora 33

#135

Post by cmurf »

WGH_ wrote: post #125
cmurf wrote: post #115

Earlier quoted context omitted.

If it's reproducible, my suggestion is to strace the rm command and find out what it's doing that's taking so long; and post it to the mailing list: https://btrfs.wiki.kernel.org/index.php/Btrfs_mailing_list

Recording the kernel profile with perf would be more useful, I think. strace would probably only show long "unlink" syscall.

Yeah, and if perf is unrevealing, sysrq+t.

Re: Btrfs Coming to Fedora 33

#136

Post by cmurf »

codys wrote: post #134
cmurf wrote: post #117

Earlier quoted context omitted.

What is the source of "Btrfs requires a regularly run administrative task...". No file system should require this kind of regular hand holding. If it does, it's a bug, and needs to be reported and fixed, not papered over with user initiated maintenance tasks. It is acceptable to have utilities for optimizing the layout from time to time, but they should be considered optimizations, not requirements. If it becomes req…

There are whole repos[1] dedicated to scripts to perform the delicate work of running `btrfs balance` in various ways to ensure space is recovered. openSuse (a distro which notably defaults to btrfs) packages these btrfsmaintenance scripts [2], and it appears they may have included it in their default install (I can't find a list of packages). Their wiki page on disabling btrfsmaintenance [5] implies that if disabled…

openSUSE has automatic snapshots with a fairly extensive retention policy.

Kernel 3.14 is ancient. I can't bring myself to worry about it.

I think your fourth paragraph cherry pick is disingenuous. A more complete excerpt, "There’s a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18."

Fedora 33 users will be getting kernel 5.8 from day one, and 5.9 soon after release.

I doubt my case is atypical. I haven't balanced my years old non-test real world used Btrfs file systems.

I think you could switch your concern and criticism to the fact wikis get stale.

Re: Btrfs Coming to Fedora 33

#137

Post by codys »

cmurf wrote: post #136
codys wrote: post #134

Earlier quoted context omitted.

There are whole repos[1] dedicated to scripts to perform the delicate work of running `btrfs balance` in various ways to ensure space is recovered. openSuse (a distro which notably defaults to btrfs) packages these btrfsmaintenance scripts [2], and it appears they may have included it in their default install (I can't find a list of packages). Their wiki page on disabling btrfsmaintenance [5] implies that if disabled…

openSUSE has automatic snapshots with a fairly extensive retention policy. Kernel 3.14 is ancient. I can't bring myself to worry about it. I think your fourth paragraph cherry pick is disingenuous. A more complete excerpt, "There’s a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18." Fedora 3…

> openSUSE has automatic snapshots with a fairly extensive retention policy.

Ah, so Fedora is limiting it's use of snapshots to avoid the need to have balances occur? Do you have some info on what level snapshot usage has to rise to before balances are needed on a regular basis? Is Fedora using snapshots at all?

Re: Btrfs Coming to Fedora 33

#138

Post by codys »

ac29 wrote: post #110
codys wrote: post #76

Earlier quoted context omitted.

My experience with btrfs is from a few years ago, so it's possible they've changed the semantics here since then, but this might be applicable: Btrfs requires a regularly run administrative task via "btrfs balance". Balancing consolidates data between partially filled block groups, which will restore consumed space even if a block group is not entirely empty. Typically, one runs balance with a set of gradually increa…

> Btrfs requires a regularly run administrative task via "btrfs balance" This hasnt been my experience within the past few years, though I do remember it being somewhat necessary in the past. Both systems I'm running btrfs on have their free space within a few percent of the unallocated space (indicating most used blocks are fairly full and little space is wasted).

Are you using snapshots in btrfs? Perhaps trying to use this feature was what caused it to be needed for my use case.

Re: Btrfs Coming to Fedora 33

#139

Post by KaiserPro »

blaser-waffle wrote: post #94
eptcyka wrote: post #72

Earlier quoted context omitted.

I for one support Facebook's push to make more problems solvable solely by entering a mud hut and ingesting hallucinogenics.

That won't make them less evil, just less coherent.

Its only less coherent if youre not off your tits.

Re: Btrfs Coming to Fedora 33

#140

Post by cmurf »

codys wrote: post #137
cmurf wrote: post #136

Earlier quoted context omitted.

openSUSE has automatic snapshots with a fairly extensive retention policy. Kernel 3.14 is ancient. I can't bring myself to worry about it. I think your fourth paragraph cherry pick is disingenuous. A more complete excerpt, "There’s a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18." Fedora 3…

> openSUSE has automatic snapshots with a fairly extensive retention policy. Ah, so Fedora is limiting it's use of snapshots to avoid the need to have balances occur? Do you have some info on what level snapshot usage has to rise to before balances are needed on a regular basis? Is Fedora using snapshots at all?

There is no automatic snapshotting regime on Fedora.

There's no direct correlation between having many snapshots, and needing to balance. The once per month balance used in openSUSE is to preempt or reduce the chances of out of space error in one type of chunk (block group of extents), while there remains significant free space in another type of chunk.

Chunks are created dynamically, and are either type metadata or data (also system but it can be ignored). Different workloads have different data/metadata ratio demands, hence dynamic allocation. Snapshots are almost entirely metadata. More snapshotting means more usage of metadata.

If the pattern dramatically changes, the ratio also changes, and the dynamic allocation can alter course. Except when the disk is fully allocated. In that case, heavy metadata writes will completely fill metadata chunks, and ENOSPC even though there's still unused space in data chunks.

A filtered balance can move extents from one chunk to another, and once a chunk is empty, it can be deallocated. That unallocated space can now be allocated into a different type of chunk, thus avoiding ENOSPC. Or at least when ENOSPC happens, there's essentially no free space in either chunk type, at the same time. A "true" ENOSPC.

There are all kinds of mitigations for the problem in newer kernels. And in my opinion it's better to not paper over problems, but fixing the remaining edge cases.

Post reply on HN