Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS fans, rejoice – RAIDz expansion will be a thing soon

arstechnica.com

Post reply on HN
Print view View original

161–170 of 198 posts

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#161

Post by tomatocracy »

tobias3 wrote: post #156
tomatocracy wrote: post #155

Earlier quoted context omitted.

Some of these are fair points but zfsonlinux/OpenZFS has had O_DIRECT since 0.8.x.

ZFSOnLinux just ignores the O_DIRECT flag if I remember correctly. Granted, this is what btrfs should do by default as well since there is an ugly issue where software can modify the O_DIRECT buffer after it was submitted causing btrfs checksum errors even though nothing was corrupted (and there is nothing to be done about it except disabling O_DIRECT or creating a buffer copy).

Yes (and as you mention this is the right thing to do) but I think it means that the Linux kernel will bypass the page cache which is still useful.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#162

Post by curt15 »

the8472 wrote: post #32
d33lio wrote: post #2

I'll believe it when I see it, why anyone uses BTRFs (UnRaid or any other form of software raid that isn't ZFS) is still beyond me. At least when we're not talking SSD's ;) ZFS is incredible, curious to mess around with these new features!

btrfs does have some advantages over zfs - no data duplicated between page cache and arc - no upgrade problems on rolling distros - balance allows restructuring the array - offline dedup, no need for huge dedup tables - ability to turn off checksumming for specific files - O_DIRECT support - reflink copy - fiemap - easy to resize

>- ability to turn off checksumming for specific files

This is something that has always confused me. BTRFS users are always advised to disable copy-on-write (thus preventing checksumming or compression) for VM images or database files to avoid massive performance hits from fragmentation. Even Facebook still stores its databases mostly on XFS filesystems. However, the ZFS community seems to indicate that you can achieve reasonable performance for databases and VMs just by tuning the recordsize (e.g. https://pg.uptrace.dev/zfs/). How does ZFS mitigate the problems from fragmentation?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#163

Post by jen20 »

rincebrain wrote: post #136
sneak wrote: post #83

Earlier quoted context omitted.

Is there a CLA for OpenZFS/ZoL? I don't believe there is, so I don't think Oracle can unilaterally relicense it.

Even if there were a CLA for OpenZFS, it wouldn't affect Oracle's inability to relicense the whole thing. They could relicense their codebase, of course, but the number of changes that have happened since they diverged is not small.

A CLA and copyright assignment was how Oracle were able to make (now Oracle) ZFS proprietary again in the first place. As you say though, OpenZFS and Oracle ZFS have diverged quite a bit, and most of the world is now based around the OpenZFS variant that acts as the upstream for Linux, FreeBSD and even Windows variants.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#165

Post by magicalhippo »

nwmcsween wrote: post #107

I'm starting to get concerned about the ZFS issue list, there are a ton of gotchas hiding in using OpenZFS that will cause data loss: * Swap on ZVOL (data loss) * Hardlocking when removing ZIL (this has caused dataloss for us)

> Hardlocking when removing ZIL

You cannot remove the ZIL, it's an integral part of ZFS. I assume you mean a SLOG[1].

That said, do you have an issue link? I'm curious. I know people have had some issues with importing pools with a bad or missing SLOG device, though most couldn't be reproduced.

[1]: https://www.ixsystems.com/blog/o-slog-not-slog-best-configur...

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#166

Post by nix23 »

WastingMyTime89 wrote: post #160
paulmd wrote: post #133

Earlier quoted context omitted.

There is also a linear degradation of metadata/blockinfo performance (in the order of the number of rewrites) due to the need to check multiple possible locations for the block pointer. In practice this probably just means more RAM utilization but... eh. Even as a home user, I've become convinced the juice is not worth the squeeze. For serious use, just buy four drives at a time and upgrade four drives at a time. Tha…

> Meanwhile ZFS is far far better than the alternatives (btrfs) in terms of data integrity and reliability. No, it's not. Btrfs is nowadays perfectly reliable as long as you avoid the in development features which are all explicitly marked as in development and will warn you. That's why Facebook uses it in production. It also has some nice advantage for home use. Btrfs pools are a lot simpler to manage and have less…

I used/use SLES for a long time, and there is a strong recommendation from SUSE itself how you should btrfs with SLES.

-Just use it for the OS, XFS for data

-Just use it in mirror configuration

-Don't touch anything else

-Make Snapshots and delete them is stable

With those points, i too had never a problem.

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#167

Post by nix23 »

Datagenerator wrote: post #93
rektide wrote: post #78

Earlier quoted context omitted.

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

Netflix has been using ZFS in production for many years now. Unnamed research companies are using ZFS moving PB's of data. NetApp is FreeBSD based and was on the forefront of what we now call ZFS. I'm totally biased, designed many production critical systems with ZFS at it's core in one way or another. The power of ZFS send and receive function is tremendous to say the least, it beats any file based synchronizing met…

>Netflix has been using ZFS in production

User of freebsd myself but that is BS, Netflix uses freebsd and UFS2 on the openconnect device.

https://openconnect.netflix.com/en/

You watch youtube about netflix they state clearly that they use ufs.

I "think" it's that one

https://www.youtube.com/watch?v=veQwkG0WdN8

Microservice-backend is linux/aws

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#168

Post by nix23 »

rektide wrote: post #130
nullwarp wrote: post #124

Earlier quoted context omitted.

Send/Recv and snapshots by far my favorite features, I keep snapshots of my root drive on my NAS so if my main m2 drive dies in my desktop, i can just boot a live cd and send/recv to my new drive and reboot and i'm back where i left off. I also keep a clean copy of an installed debian install snapshot on my NAS so i can just send it to new machines rather than run through the whole setup. works great.

Snapshots & send-receive are my favorite BTRFS features too! I too love snapshotting Debian installs & send'ing them to new machines for atomic updates, or to build live usb images. I've been using Pottering's "Revisiting How We Put Together Linux Systems" naming system for my subvolumes, and that's worked fairly well[1]. I even did some fiddling around with enhancing Debian-live to load btrfs into ram a while back,…

>we should be cooperative & boosting each other.

Is that why you sound like a sour apple?

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#169

Post by nix23 »

nickik wrote: post #149
rektide wrote: post #78

Earlier quoted context omitted.

No matter what happens, people will seemingly forever declare BTRFS is not as stable and not as safe. There's a status page that details what BTRFS thinks of itself[1], and I doubt any of the many people docking BTRFS have read or know or care what that page says. There is one issue still being worked out to completion, a "write hole" problem, involving two separate failures, an unplanned/power-loss shut-down, follow…

> it feels to me like there is a persistent & vocal small group of people who seemingly either have some agenda that makes them not wish to consider BTRFS Look here is the problem. The BTRFS people declared multiple time that it is stable. For a few years it was repeatably 'BTRFS' is stable now if you use it in such an such a away. But then it destroyed a drive. Then it a few years later and now it was actually stabl…

If a filesystem looses your data, you never use it again.

Kirk McKusick creator of UFS/2

Re: ZFS fans, rejoice – RAIDz expansion will be a thing soon

#170

Post by j1elo »

jpeloquin wrote: post #147
j1elo wrote: post #41

Earlier quoted context omitted.

What about if you were just starting today, with 0 knowledge about basically anything related to storage and how to do it right? That's my case, I'm learning before setting up a cheap home lab and a NAS, and I'm wondering if biting into ZFS is just the best option that I have given today's ecosystem.

I was in the same place 6 or 7 years ago. Due to indecision, I ended up using btrfs, zfs, and mdadm (technically, Synology hybrid raid) on various devices. They all work, more or less. Looking back, the lessons that come to mind are: - Always have 2 backups (not counting the primary copy), at least 1 "cold" (inaccessible without human intervention) and at least 1 offsite. Backup frequently and retain old backups. Wit…

Thank you. I need to start small, otherwise I feel overwhelmed by too many moving pieces to keeo in mind and plan for.

So I'm starting small, from powering up a ThinkCentre M910 I had laying around, with an internal disk that can be used to store backups. I have 0 need for performance so my idea was to extend storage with an external USB3 HD enclosure. For now, I don't have the space nor the machine where to install dual hard disks for building a decent RAID. Time will tell.

Post reply on HN