Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS 2.3 released with ZFS raidz expansion

github.com

Post reply on HN
Print view View original

241–250 of 331 posts

Re: ZFS 2.3 released with ZFS raidz expansion

#241

Post by phs2501 »

fourfour3 wrote: post #89
lousken wrote: post #74

Earlier quoted context omitted.

as far as stability goes, btrfs is used by meta, synology and many others, so I wouldn't say it's not stable, but some features are lacking

Do Synology actually use the multi-device options of btrfs, or are they using linux softraid + lvm underneath? I know Synology Hybrid RAID is a clever use of LVM + MD raid, for example.

I believe Synology runs btrfs on top of regular mdraid + lvm, possibly with patches to let btrfs checksum failures reach into the underlying layers to find the right data to recover.

Related blog post: https://daltondur.st/syno_btrfs_1/

Re: ZFS 2.3 released with ZFS raidz expansion

#242

Post by jeltz »

lousken wrote: post #74
GuB-42 wrote: post #73

Earlier quoted context omitted.

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

as far as stability goes, btrfs is used by meta, synology and many others, so I wouldn't say it's not stable, but some features are lacking

It is possible to corrupt the file system from user space as a normal user with Btrfs. The PostgreSQL devs found that when working on async IO. And as fer as I know that issue has not been fixed.

https://www.postgresql.org/message-id/CA%2BhUKGL-sZrfwcdme8j...

Re: ZFS 2.3 released with ZFS raidz expansion

#243

Post by NoMoreNicksLeft »

poisonborz wrote: post #161
bayindirh wrote: post #127

Earlier quoted context omitted.

> How come that Windows still uses a 32 year old file system? Simple. Because most of the burden is taken by the (enterprise) storage hardware hosting the FS. Snapshots, block level deduplication, object storage technologies, RAID/Resiliency, size changes, you name it. Modern storage appliances are black magic, and you don't need much more features from NTFS. You either transparently access via NAS/SAN or store your…

So private consumers should just pay cloud subscription if they want safer/modern data storage for their PC? (without NAS)

Having a NAS is life-changing. Doesn't have to be some large 20-bay monstrosity, just something that will give you redundancy and has an ethernet jack.

Re: ZFS 2.3 released with ZFS raidz expansion

#244

Post by jeltz »

ryao wrote: post #234
bayindirh wrote: post #129

Earlier quoted context omitted.

> Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. However, ext4 and XFS are much more simpler and performant than BTRFS & ZFS as root drives on personal systems and small servers. I personally won't use either on a single disk system as root FS, regardless of how fast my storage subsystem is.

ZFS will outscale ext4 in parallel workloads with ease. XFS will often scale better than ext4, but if you use L2ARC and SLOG devices, it is no contest. On top of that, you can use compression for an additional boost. You might also find ZFS outperforms both of them in read workloads on single disks where ARC minimizes cold cache effects. When I began using ZFS for my rootfs, I noticed my desktop environment became mo…

Not on most database workloads. There zfs does not scale very well.

Re: ZFS 2.3 released with ZFS raidz expansion

#245

Post by BodyCulture »

stephen_g wrote: post #110
GuB-42 wrote: post #73

Earlier quoted context omitted.

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS compon…

I’m interested to know what ‘full integration’ does look like, I use ZFS in Proxmox (Debian-based) and it’s really great and super solid, but I haven’t used ZFS in more vanilla Linux distros. Does Proxmox have things that regular Linux is missing out on, or are there shortcomings and things I just don’t realise about Proxmox?

You probably don’t realise how important encryption is.

It’s still not supported by Proxmox, yes, you can do it yourself somehow but you are alone then and miss features and people report problems with double or triple file system layers.

I do not understand how they have not encryption out of the box, this seems to be a problem.

Re: ZFS 2.3 released with ZFS raidz expansion

#246

Post by GeorgeTirebiter »

snvzz wrote: post #20
jdboyd wrote: post #6

Earlier quoted context omitted.

The first 4 seem like really big deals.

The fifth is also, once you consider non-ascii names.

Could someone show a legit reason to use 1000-character filenames? Seems to me, when filenames are long like that, they are actually capturing several KEYS that can be easily searched via ls & re's. e.g.

2025-Jan-14-1258.93743_Experiment-2345_Gas-Flow-375.3_etc_etc.dat

But to me this stuff should be in metadata. It's just that we don't have great tools for grepping the metadata.

Heck, the original Macintosh FS had no subdirectories - they were faked by burying subdirectory names in the (flat filesysytem) filename. The original Macintosh File System (MFS), did not support true hierarchical subdirectories. Instead, the illusion of subdirectories was created by embedding folder-like names into the filenames themselves.

This was done by using colons (:) as separators in filenames. A file named Folder:Subfolder:File would appear to belong to a subfolder within a folder. This was entirely a user interface convention managed by the Finder. Internally, MFS stored all files in a flat namespace, with no actual directory hierarchy in the filesystem structure.

So, there is 'utility' in "overloading the filename space". But...

Re: ZFS 2.3 released with ZFS raidz expansion

#247

Post by kevinmgranger »

BodyCulture wrote: post #245
stephen_g wrote: post #110

Earlier quoted context omitted.

I’m interested to know what ‘full integration’ does look like, I use ZFS in Proxmox (Debian-based) and it’s really great and super solid, but I haven’t used ZFS in more vanilla Linux distros. Does Proxmox have things that regular Linux is missing out on, or are there shortcomings and things I just don’t realise about Proxmox?

You probably don’t realise how important encryption is. It’s still not supported by Proxmox, yes, you can do it yourself somehow but you are alone then and miss features and people report problems with double or triple file system layers. I do not understand how they have not encryption out of the box, this seems to be a problem.

I'm not sure about proxmox, but ZFS on Linux does have encryption.

Re: ZFS 2.3 released with ZFS raidz expansion

#248

Post by BodyCulture »

scrp wrote: post #2

After years in the making ZFS raidz expansaion is finally here. Major features added in release: - RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime. - Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality. - Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where cachi…

How well tested is this in combination with encryption?

Is the ZFS team handling encryption as a first class priority at all?

ZFS on Linux inherited a lot of fame from ZFS on Solaris, but everyone using it in production should study the issue tracker very well for a realistic impression of the situation.

Re: ZFS 2.3 released with ZFS raidz expansion

#249

Post by [deleted] »

xattt wrote: post #131
ryao wrote: post #86

Earlier quoted context omitted.

There is a third party port here: https://openzfsonosx.org/wiki/Main_Page It was actually the NetApp lawsuit that caused problems for Apple’s adoption of ZFS. Apple wanted indemnification from Sun because of the lawsuit, Sun’s CEO did not sign the agreement before Oracle’s acquisition of Sun happened and Oracle had no interest in granting that, so the official Apple port was cancelled. I heard this second hand years…

That’s a shame re: NetApp/ZFS. While third-party ports are great, they lack deep integration that first-party support would have brought (non-kludgy Time Machine which is technically fixed with APFS).

[deleted]

Re: ZFS 2.3 released with ZFS raidz expansion

#250

Post by p_l »

BodyCulture wrote: post #248
scrp wrote: post #2

After years in the making ZFS raidz expansaion is finally here. Major features added in release: - RAIDZ Expansion: Add new devices to an existing RAIDZ pool, increasing storage capacity without downtime. - Fast Dedup: A major performance upgrade to the original OpenZFS deduplication functionality. - Direct IO: Allows bypassing the ARC for reads/writes, improving performance in scenarios like NVMe devices where cachi…

How well tested is this in combination with encryption? Is the ZFS team handling encryption as a first class priority at all? ZFS on Linux inherited a lot of fame from ZFS on Solaris, but everyone using it in production should study the issue tracker very well for a realistic impression of the situation.

Main issue with encryption is occasional attempts by certain (specific) Linux kernel developer to lockout ZFS out of access to advanced instruction set extensions (far from the only weird idea of that specific developer).

The way ZFS encryption is layered, the features should be pretty much orthogonal from each other, but I'll admit that there's a bit of lacking with ZFS native encryption (though mainly in upper layer tooling in my experience rather than actual on-disk encryption parts)

Post reply on HN