Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Examining Btrfs

arstechnica.com

Post reply on HN
Print view View original

51–60 of 112 posts

Re: Examining Btrfs

#51

Post by dijit »

jcalvinowens wrote: post #46
dijit wrote: post #45

Earlier quoted context omitted.

You’re not correct on a number of fronts here. ZFS is not tied to the memory allocator in Solaris, I’m very unclear how you could have made that mistake. 128bits might feel wasteful, but you’re likely never going to have the problems FAT32 has had- it’s designed for the next 100 years- which has drawbacks but prevent the kinds of ugly hacks that NTFS has gone through to maintain forward compatibility (rolling on disk…

> ZFS is not tied to the memory allocator in Solaris, I’m very unclear how you could have made that mistake > And yes, the 1G per TB is for the in memory block map, which is a requirement for deduplucation on the default block size. You actually answered your own question: it's antiquated because of the assumption it is possible to get contiguous RAM for that mapping. On modern Linux, that is only possible if you res…

The Solaris memory allocator is not contiguous either though

Re: Examining Btrfs

#52

Post by detaro »

jcalvinowens wrote: post #46
dijit wrote: post #45

Earlier quoted context omitted.

You’re not correct on a number of fronts here. ZFS is not tied to the memory allocator in Solaris, I’m very unclear how you could have made that mistake. 128bits might feel wasteful, but you’re likely never going to have the problems FAT32 has had- it’s designed for the next 100 years- which has drawbacks but prevent the kinds of ugly hacks that NTFS has gone through to maintain forward compatibility (rolling on disk…

> ZFS is not tied to the memory allocator in Solaris, I’m very unclear how you could have made that mistake > And yes, the 1G per TB is for the in memory block map, which is a requirement for deduplucation on the default block size. You actually answered your own question: it's antiquated because of the assumption it is possible to get contiguous RAM for that mapping. On modern Linux, that is only possible if you res…

> it's antiquated because of the assumption it is possible to get contiguous RAM for that mapping.

source on that? (and even if that's the case, wouldn't that be an implementation detail? E.g. implementations can spill it to L2ARC, but somehow not to different memory locations?)

Re: Examining Btrfs

#53

Post by bluGill »

COGlory wrote: post #36

>Moving beyond the question of individual disk reliability, btrfs-raid1 can only tolerate a single disk failure, no matter how large the total array is. The remaining copies of the blocks that were on a lost disk are distributed throughout the entire array—so losing any second disk loses you the array along with it. (This is in contrast to RAID10 arrays, which can survive any number of disk failures as long as no two…

Only in small arrays. In larger zfs systems RAID-1 you divide the disks into chunks of drives - 5-7 is typical. If you have 10 drives, any one can fail without issues, but even with that one failure, a second can fail without losing data 50% of the time. This puts the odds more in your favor. If you are like the typical single person 5 disks is enough and so this doesn't matter at all. However if you a larger organization with several hundred disks odds are you have more than one failed at a time on a regular basis, but the odds that more than one failed drive is from the same chunk is still low enough to not worry too much.

I personally have zfs-z2 with 6 disks in my NAS. This any 2 drives can fail. If I want to expand my NAS I would need 6 more disks, but not only could any 2 fail, many combinations of up to 4 can fail without issues.

If you have a small number of drives this doesn't matter, but BTRFS-raid1 is not acceptable for large number of disks.

Re: Examining Btrfs

#54

Post by mst »

COGlory wrote: post #36

>Moving beyond the question of individual disk reliability, btrfs-raid1 can only tolerate a single disk failure, no matter how large the total array is. The remaining copies of the blocks that were on a lost disk are distributed throughout the entire array—so losing any second disk loses you the array along with it. (This is in contrast to RAID10 arrays, which can survive any number of disk failures as long as no two…

> This feels a little disingenuous to me. This is a distinctly RAID-1 problem. This doesn't feel like a BTRFS problem. Everyone knows RAID-1 = 1 drive loss.

The point he's making is that classic RAID-1 is pairs of disks, of which you can lose one of the two, whereas btrs-raid1's (admittedly very neat) feature to bolt multiple disks together means you can end up with a situation where you can only lose one of N drives, which is significantly riskier.

> There are many use cases for RAID-0, but the author is acting like this is some kind of problem with BTRFS.

I read the "anything specific" as them being clear that it's not remotely btrfs specific and is, indeed, exactly a limitation of RAID-0.

Plus over the years I've definitely run into a lot more cases where people have misused RAID-0 and ended up with an unexpected virtual woodshipper than cases where they were actually right to use it (even those do absolutely exist), which does have a tendency to colour one's experience. I think "Avoid" is the safest advice to give in a general-audience article, people who will genuinely get advantages out of RAID-0 in spite of the failure modes are likely to be able to figure that out themselves anyway.

Re: Examining Btrfs

#55

Post by jcalvinowens »

dijit wrote: post #51
jcalvinowens wrote: post #46

Earlier quoted context omitted.

> ZFS is not tied to the memory allocator in Solaris, I’m very unclear how you could have made that mistake > And yes, the 1G per TB is for the in memory block map, which is a requirement for deduplucation on the default block size. You actually answered your own question: it's antiquated because of the assumption it is possible to get contiguous RAM for that mapping. On modern Linux, that is only possible if you res…

The Solaris memory allocator is not contiguous either though

This is a whole rabbit hole... but essentially, Linux uses a naive buddy allocator for physical RAM, and Solaris was more interesting.

Re: Examining Btrfs

#56

Post by topspin »

nimbius wrote: post #37
DiabloD3 wrote: post #19

Brtfs only exists as Oracle's "replacement" for ZFS, to hedge their bets in case they couldn't buy Sun. Oracle bought sun in 2010, that has been 12 years. They merged the Btrfs and ZFS teams, they then worked on ZFS entirely with no Btrfs work, and then basically reassigned, fired, or otherwise dissolved the ZFS team when it became clear that OpenZFS (Sun's chosen steward for ZFS) could not be swayed. Btrfs has had n…

>Btrfs has had no real work done on it since then changelog seems to refute this point entirely. https://btrfs.wiki.kernel.org/index.php/Changelog >anyone that is actually serious about data storage has moved to ZFS sometimes data storage doesnt have to scale, and thats okay. there are still plenty of use cases for lvm/xfs/mdadm in environments that dont need Google levels of service. im "serious" about data storage…

> redhat intentionally threw backing into XFS to make it functional with Ceph.

Can you elaborate on that? Is the about pre-bluestore backing of Ceph or... something else?

Re: Examining Btrfs

#57

Post by [deleted] »

jcalvinowens wrote: post #35
DiabloD3 wrote: post #19

Brtfs only exists as Oracle's "replacement" for ZFS, to hedge their bets in case they couldn't buy Sun. Oracle bought sun in 2010, that has been 12 years. They merged the Btrfs and ZFS teams, they then worked on ZFS entirely with no Btrfs work, and then basically reassigned, fired, or otherwise dissolved the ZFS team when it became clear that OpenZFS (Sun's chosen steward for ZFS) could not be swayed. Btrfs has had n…

This is one of the most laughably ignorant posts I've ever seen on the internet. > Btrfs has had no real work done on it since then, and anyone that is actually serious about data storage has moved to ZFS, or has left POSIX filesystems entirely Btrfs is the most actively developed filesystem in the kernel, both by LOC changed and the number of contributors. Facebook publicly stated that they run BTRFS on every webser…

[deleted]

Re: Examining Btrfs

#59

Post by sumtechguy »

uniqueuid wrote: post #24
zionic wrote: post #20

Earlier quoted context omitted.

I’ve been incredibly happy with my Synology products FWIW. BTRFS/SHR have been rock solid.

The hardware is very nice indeed, and the software is smooth as well. What has irritated me in the past was that they sometimes took very long to fix vulnerabilities [1]. [1] https://www.cvedetails.com/vendor/11138/Synology.html

What is getting me is the EOL bits they have done with some of their HW. It is a NAS. I can see EOL on parts of the stuff they add. But the fileshare bits? That is really the only reason I bought the thing. Does anyone know any alt distros to replace theirs?

Re: Examining Btrfs

#60

Post by bluGill »

derbOac wrote: post #39

Can someone recommend a good overview of modern filesystems that covers performance and licensing issues? I liked this article but was left with the question "so what else do you use?" It's been awhile (too long really) since I really tried to read up on this area. Last time I did I concluded that it was sort of a mess but lots of things were still in progress.

Use ZFS if you can. However for legal reasons it is hard to use on linux, which pushes you to FreeBSD (I'm not sure about the others). FreeBSD can replace linux, but you will find a number of places where the software you use assumes linux and so things don't work right.

If you are doing NAS all on your own, then don't hesitate to run to FreeBSD. However if you want to a not do it yourself NAS it is a little harder. The most popular/polished commercial NAS is synology based on BTRFS. I personally use TrueNAS which is a great file server, but synology has some nice integration to photo and media applications and so I'm thinking about switching.

Post reply on HN