Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

121–130 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#121

Post by SkyMarshal »

wyager wrote: post #58
nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

ZFS RAID is the best RAID implementation in many respects. Hardware RAID is bad at actually fixing errors on disk (as opposed to just transparently correcting) and surfacing errors to the user. BTRFS is frequently not considered stable enough for production usage. ZFS has dozens of useful features besides RAID. Transparent compression, instant atomic snapshots, incremental snapshot sync, instant cloning of file syste…

>BTRFS is frequently not considered stable enough for production usage.

I'm not sure this is still true, especially after Facebook deployed it on millions of servers.

https://btrfs.wiki.kernel.org/index.php/Production_Users

https://btrfs.wiki.kernel.org/index.php/Production_Users

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#122

Post by bashinator »

SkyMarshal wrote: post #121
wyager wrote: post #58

Earlier quoted context omitted.

ZFS RAID is the best RAID implementation in many respects. Hardware RAID is bad at actually fixing errors on disk (as opposed to just transparently correcting) and surfacing errors to the user. BTRFS is frequently not considered stable enough for production usage. ZFS has dozens of useful features besides RAID. Transparent compression, instant atomic snapshots, incremental snapshot sync, instant cloning of file syste…

>BTRFS is frequently not considered stable enough for production usage. I'm not sure this is still true, especially after Facebook deployed it on millions of servers. https://btrfs.wiki.kernel.org/index.php/Production_Users https://btrfs.wiki.kernel.org/index.php/Production_Users

So what you’re saying is, btrfs works if you can afford an ops/sre team on par with Facebook’s?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#123

Post by nullwarp »

rektide wrote: post #78
nimbius wrote: post #53

this might sound like a troll comment but its coming from someone with almost zero experience with raid. What is the purpose of ZFS in 2021 if we have hardware RAID and linux software RAID? BTRFS does RAID too. Why would people choose ZFS in 2021 if both Oracle and Open Source users have 2 competing ZFS? are they interoperable?

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

I've lost data in BTRFS setups each of the three times I've given it a try over the course of 6 or so years. Root drive just became unrecoverable. These are all single disk setups

Meanwhile I've been running ZFS for close to the same time and have never lost anything.

I get it's an anecdotal view point but that's a very hard reputation to rebuild for BTRFS.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#124

Post by nullwarp »

Datagenerator wrote: post #93
rektide wrote: post #78

Earlier quoted context omitted.

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

Netflix has been using ZFS in production for many years now. Unnamed research companies are using ZFS moving PB's of data. NetApp is FreeBSD based and was on the forefront of what we now call ZFS. I'm totally biased, designed many production critical systems with ZFS at it's core in one way or another. The power of ZFS send and receive function is tremendous to say the least, it beats any file based synchronizing met…

Send/Recv and snapshots by far my favorite features, I keep snapshots of my root drive on my NAS so if my main m2 drive dies in my desktop, i can just boot a live cd and send/recv to my new drive and reboot and i'm back where i left off.

I also keep a clean copy of an installed debian install snapshot on my NAS so i can just send it to new machines rather than run through the whole setup. works great.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#125

Post by GauntletWizard »

uniqueuid wrote: post #24

The article is a great example of all the somewhat surprising peculiarities in ZFS. For example, the conversion will keep the stripe width and block size, meaning your throughput of existing data won't improve. So it's not quite a full re-balance. Other fun things are the flexible block sizes and their relation to the size you're writing and compression ... Chris Siebenmann has written quite a bit about it ( https://…

I'm immensely pleased with the approach they've chosen, because it offers the possibility of improving your stripe size, without requiring commitment to. New blocks are written with the new stripe size, so all you have to do to improve your stripe width is rewrite all of your files (and deal with the snapshot churn that will cause). It also offers the tempting possibility of the reverse, where you can create new files with stripes and then rewrite all your data to reduce capacity.

I used to work at Isilon, and the design of this feature looks very familiar to me: That's a good thing. I'm excited for the possibilities for ZFS going forward. I hope they'll come up with a restriper tool that will allow you to rewrite all of your files with the new block layout without breaking snapshots, etc, but either way, this is a huge advancement for my own personal use of ZFS.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#126

Post by plus »

znpy wrote: post #94
1980phipsi wrote: post #14

This will be very useful! TIL FreeNAS is now TrueNAS.

Actually there's more! A new version of TrueNAS is in the works, it's called TrueNAS scale and it's going to be Linux-based (no more FreeBSD). I'm frankly happy, because TrueNAS is great as a NAS operating system but I really wanted to run containers where my storage is and having to run a VM adds a really unnecessary overhead (plus, it's another machine to manage)

TrueNAS supports creation and management of FreeBSD jails through its WebUI.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#127

Post by Johnny555 »

I was disappointed by the lack of RAIDZ2 resize when I built my ZFS fileserver, but it turns out that my data growth is slower than the growth in size of HDD's, so I just replace the drives every 4 or 5 years and copy the data over. Now drives are so big that I might just go with mirroring instead of RAIDZ2

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#129

Post by ori_b »

znpy wrote: post #94
1980phipsi wrote: post #14

This will be very useful! TIL FreeNAS is now TrueNAS.

Actually there's more! A new version of TrueNAS is in the works, it's called TrueNAS scale and it's going to be Linux-based (no more FreeBSD). I'm frankly happy, because TrueNAS is great as a NAS operating system but I really wanted to run containers where my storage is and having to run a VM adds a really unnecessary overhead (plus, it's another machine to manage)

Ah, so Linux is approaching feature parity with FreeBSD jails?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#130

Post by rektide »

nullwarp wrote: post #124
Datagenerator wrote: post #93

Earlier quoted context omitted.

Netflix has been using ZFS in production for many years now. Unnamed research companies are using ZFS moving PB's of data. NetApp is FreeBSD based and was on the forefront of what we now call ZFS. I'm totally biased, designed many production critical systems with ZFS at it's core in one way or another. The power of ZFS send and receive function is tremendous to say the least, it beats any file based synchronizing met…

Send/Recv and snapshots by far my favorite features, I keep snapshots of my root drive on my NAS so if my main m2 drive dies in my desktop, i can just boot a live cd and send/recv to my new drive and reboot and i'm back where i left off. I also keep a clean copy of an installed debian install snapshot on my NAS so i can just send it to new machines rather than run through the whole setup. works great.

Snapshots & send-receive are my favorite BTRFS features too! I too love snapshotting Debian installs & send'ing them to new machines for atomic updates, or to build live usb images. I've been using Pottering's "Revisiting How We Put Together Linux Systems" naming system for my subvolumes, and that's worked fairly well[1]. I even did some fiddling around with enhancing Debian-live to load btrfs into ram a while back, so I could atomic update, de-mount the boot drive, &c[2].

Oh you are talking about ZFS, aren't you. ;)

Playing the "who came up with it first" game doesn't particularly interest me. Again, it feels like a spirit of competition when to me, that seems like a bad spirit: we should be cooperative & boosting each other. We're both open source, we're both trying to make civilization possible & to share greatnesses.

[1] http://0pointer.net/blog/revisiting-how-we-put-together-linu...

[2] https://github.com/rektide/debian-live-boot

Post reply on HN