Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Btrfs Allocator Hints

lwn.net

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Btrfs Allocator Hints

#51

Post by mdedetrich »

homebrewer wrote: post #36
gavinsyancey wrote: post #33

Earlier quoted context omitted.

My btrfs filesystem has been slowly eating my data for a while; large files will find their first 128k replaced with all nulls. Rewriting it will sometimes fix it temporarily, but it'll revert back to all nulls after some time. That said, this might be my fault for using raid6 for data and trying to replace a failing disk a while ago.

raid 5/6 is completely broken and there's not much interest in fixing it — nobody who's willing to pay for its development (which includes Facebook, SUSE, Oracle, and WD) uses raid 5/6; you shouldn't have been running it in the first place. I understand it's basically blaming the victim, but doing at least some research on the filesystem before starting to use it is a good idea in any case. https://btrfs.readthedocs.…

iirc, btrfs has fixed the issues with raid 5/6 but it requires a breaking change to the on disk format which means you have to create an entirely new partition and copy the data over (you cannot update an existing partition to it). This new on disk format also needs its own testing.

Your point raid 5/6 not being tested heavily by actual users is entirely on point, those enterprise heavy users are only running RAID 10 like configurations.

If you want RAID 5/6, just use ZFS as they have solved all of these issues. I don't know if its due to sheer luck or maybe the fact is that Sun at its time was actually running RAID 5/6 in production (hard drives were not as cheap back then as they are now)?

Re: Btrfs Allocator Hints

#52

Post by mdedetrich »

riku_iki wrote: post #41
theamk wrote: post #40

Earlier quoted context omitted.

The reliability is about variety of workloads, not amount of data or throughput. It's easy to write a filesystem which works well in the ideal case, it's the bad or unusual traffic patters which cause problem. For all that I know maybe that btrfs complete failure was because of kernel crash caused by bad USB hardware. Or there was a cosmic ray hitting memory chip. But you know who's fault is it? It's btrfs's one. Oth…

> but as you may guess we did not use btrfs ever again. there are scenarious where btrfs is currently can't be replaced: high performance + data compression.

OpenZFS does a better job here, at least if you can deal with an out of tree filesystem.

Re: Btrfs Allocator Hints

#53

Post by riku_iki »

mdedetrich wrote: post #52
riku_iki wrote: post #41

Earlier quoted context omitted.

> but as you may guess we did not use btrfs ever again. there are scenarious where btrfs is currently can't be replaced: high performance + data compression.

OpenZFS does a better job here, at least if you can deal with an out of tree filesystem.

actually, my personal benchmarks and multiple accounts in internet say it is much slower than btrfs under the load.

Re: Btrfs Allocator Hints

#54

Post by mdedetrich »

riku_iki wrote: post #53
mdedetrich wrote: post #52

Earlier quoted context omitted.

OpenZFS does a better job here, at least if you can deal with an out of tree filesystem.

actually, my personal benchmarks and multiple accounts in internet say it is much slower than btrfs under the load.

For smaller disk setups possibly but with large enough scale ZFS ends up beating out btrfs.

Re: Btrfs Allocator Hints

#55

Post by riku_iki »

mdedetrich wrote: post #54
riku_iki wrote: post #53

Earlier quoted context omitted.

actually, my personal benchmarks and multiple accounts in internet say it is much slower than btrfs under the load.

For smaller disk setups possibly but with large enough scale ZFS ends up beating out btrfs.

I test on 2TB datasets. Do you have any specific pointers which would support your claim?

Re: Btrfs Allocator Hints

#56

Post by mdedetrich »

hyperman1 wrote: post #42

I feel a bit lost here. In the good old days, I ran ext2/ext3/ext4 and forgot about it, or Reiserfs if I felt fancy (and which was great until it wasn't). Now, there is a cambrian explosion going on. Ext4, xfs, btrfs,bcachefs, zfs. They each have their pros and cons, and it takes a while before you find out you run into an expensive limit. E.g. Ext3/4 is good, until it ran out of inodes. ZFS is good, but has only 1 p…

FDE with ZFS is kind of fighting the way things are meant to be done with ZFS. ZFS allows encryption on a per dataset/zvol basis which is the officially recommended way to do encryption (see https://arstechnica.com/gadgets/2021/06/a-quick-start-guide-...)

Re: Btrfs Allocator Hints

#57

Post by mdedetrich »

riku_iki wrote: post #55
mdedetrich wrote: post #54

Earlier quoted context omitted.

For smaller disk setups possibly but with large enough scale ZFS ends up beating out btrfs.

I test on 2TB datasets. Do you have any specific pointers which would support your claim?

I have 100+TB datasets and with a large enough SSD/RAM for L1/L2 arc ZFS edges out.

Hell even the compression algorithm that ZFS has uses/has access to (LZ4) is faster than what btrfs uses and with enough IO that matters.

Re: Btrfs Allocator Hints

#58

Post by riku_iki »

mdedetrich wrote: post #57
riku_iki wrote: post #55

Earlier quoted context omitted.

I test on 2TB datasets. Do you have any specific pointers which would support your claim?

I have 100+TB datasets and with a large enough SSD/RAM for L1/L2 arc ZFS edges out. Hell even the compression algorithm that ZFS has uses/has access to (LZ4) is faster than what btrfs uses and with enough IO that matters.

> I have 100+TB datasets and with a large enough SSD/RAM for L1/L2 arc ZFS edges out.

and your claim is that you tested it against btrfs on the same workload? Maybe you could post some specific numbers from running command from this thread? https://www.reddit.com/r/zfs/comments/1i3yjpt/very_poor_perf...

> Hell even the compression algorithm that ZFS has uses/has access to (LZ4) is faster than what btrfs uses and with enough IO that matters.

lz4 compression rate was 2x vs 7x for zstd on my data (bunch of numbers), so I didn't see point of uzing lz4 compression at all because benefits are not large enough.

Re: Btrfs Allocator Hints

#59

Post by taskforcegemini »

sandreas wrote: post #26
ghostly_s wrote: post #18

Earlier quoted context omitted.

Yes, my story with btrfs is quite similar- used it for a couple years, suddenly threw some undocumented error and refused to mount, asked about it on the dev irc channel and was told apparently it was a known issue with no solution, have fun rebuilding from backups. No suggestion that anyone was interested in documenting this issue, let alone fixing it. These same people are the only ones in the world suggesting btrf…

Ok, thank you. At least I'm not alone with this. However, I'm not too much into it and would not go as far to say it's not a recommendable project, but boy was I mad it just died without any way to recover ANYTHING :-)

probably depends on where the issue is located, but is this not normally the case with encrypted drives?

Re: Btrfs Allocator Hints

#60

Post by sandreas »

taskforcegemini wrote: post #59
sandreas wrote: post #26

Earlier quoted context omitted.

Ok, thank you. At least I'm not alone with this. However, I'm not too much into it and would not go as far to say it's not a recommendable project, but boy was I mad it just died without any way to recover ANYTHING :-)

probably depends on where the issue is located, but is this not normally the case with encrypted drives?

No. Encrypted drives should be recoverable as long as you have the valid decryption values.

I think it had nothing to do with the encryption layer... the FS layer was the problem.

Post reply on HN