Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In OpenZFS and Btrfs, everyone was just guessing

phoronix.com

Post reply on HN
Print view View original

111–120 of 172 posts

Re: In OpenZFS and Btrfs, everyone was just guessing

#111

Post by ungamedplayer »

keep_reading wrote: post #106
colechristensen wrote: post #76

Earlier quoted context omitted.

Deduplication requires an absolutely enormous amount of memory, unless you have a dataset that has a huge amount of duplication for some reason (maybe a company or school where a lot of people have their own network storage and put the same large files in?) it is usually never the right choice to enable dedup. It is cheaper to buy more disk than the memory to enable dedup (somebody could run the numbers). ZFS ARC als…

Deduplication in ZFS can be pushed onto dedicated flash storage now instead of requiring gobs of RAM.

Great channel for side attacks btw.

Re: In OpenZFS and Btrfs, everyone was just guessing

#112

Post by keep_reading »

wtcactus wrote: post #96

I see a lot of comments here using the terms ZFS and Open ZFS interchangeably. But to my understanding, these are 2 different things. - ZFS is the very stable, non open source project that only works on BSD. - Open ZFS is an effort to rewrite ZFS in a totally open and free format that now runs on Linux. This was the idea I had, but I might be incorrect.

Everything is OpenZFS now except Oracle's fork.

Re: In OpenZFS and Btrfs, everyone was just guessing

#113

Post by withinboredom »

solarkraft wrote: post #29
donmcronald wrote: post #18

I don't know about BTRFS, but with ZFS there's a lot of bad information and assumptions floating around. It's one of my favorite topics to ask AI about because it shows the obvious deficiencies in AI as it regurgitates all the bad info. So when you go to the issue tracker, it seems plausible to me that you're going to find issues where the devs can't explain what's happening because the user(s) might be doing somethi…

A lot of end users run ZFS through TrueNAS/FreeNAS and I indeed found that community to spread a lot of misinfo and un-actionable advice. I liken them to gamers lapping up any snake oil solution to get 0.1 fps more. I would suggest anyone trying to learn about ZFS to avoid it, it definitely was counter-productive to me. Reddit, random blogs and official documentation (often that by Oracle) turned out to be much bette…

Reminds me of a game (MMO) I play on the weekends. Any time someone new joins, they always ask how come frame rates are so low. Then we begin the ”set your graphic settings to high” conversation. They immediately say we are full of shit. It’s already set to low and getting 10 fps.

Eventually, someone explains: “just try it, or stop complaining.”

30s later, they exclaim: “holy ** I’m getting 60 fps now!”

Sometimes, what looks like misinformation isn’t misinformation.

TL;DR: this game switches to a software rendering pipeline on “medium” or lower. You have to set it on “high” to switch to the hardware rendering pipeline.

Re: In OpenZFS and Btrfs, everyone was just guessing

#114

Post by pnutjam »

tanelpoder wrote: post #97

Somewhat related, RedHat withdrew from the attempt to productionize BRTFS, apparently it was fully removed in RHEL 8: - https://access.redhat.com/solutions/197643 Also, there's a PDF doc (from a conference presentation), covering many layers of potential alternatives for ZFS features: - https://news.ycombinator.com/item?id=38484598 But "just" slapping all these layers on each other (configuring each individually and…

Suse has been using production btrfs for years now. https://en.opensuse.org/SDB:BTRFS

Re: In OpenZFS and Btrfs, everyone was just guessing

#115

Post by nolist_policy »

ungamedplayer wrote: post #111
keep_reading wrote: post #106

Earlier quoted context omitted.

Deduplication in ZFS can be pushed onto dedicated flash storage now instead of requiring gobs of RAM.

Great channel for side attacks btw.

Doesn't even need SSD's to open a side-channel: https://lore.kernel.org/linux-btrfs/CAKDzk=-HZardsLFH5c9HYre...

Re: In OpenZFS and Btrfs, everyone was just guessing

#116

Post by dralley »

tanelpoder wrote: post #97

Somewhat related, RedHat withdrew from the attempt to productionize BRTFS, apparently it was fully removed in RHEL 8: - https://access.redhat.com/solutions/197643 Also, there's a PDF doc (from a conference presentation), covering many layers of potential alternatives for ZFS features: - https://news.ycombinator.com/item?id=38484598 But "just" slapping all these layers on each other (configuring each individually and…

Supposedly (and I only know this because Kent has talked about it in various public conferences) Kent has a regular conference call with Red Hat engineers and there is interest from Red Hat in helping it get polished up. So perhaps in a few years bcachefs will be a supported option.

Re: In OpenZFS and Btrfs, everyone was just guessing

#117

Post by donmcronald »

angus-prune wrote: post #67
bpbp-mango wrote: post #62

Earlier quoted context omitted.

Using it through something like TrueNAS with sensible defaults is probably a better idea for the average person than trying to roll their own deployment, no?

Im not sure running TrueNAS counts as learning ZFS, which I think is GP's point. If you want to use ZFS, then TrueNAS with sensible defaults is a great way to go. If you want to learn ZFS enough to roll your own deployment, change the configuration correctly and understand the changes you make then don't start with TrueNAS, and definitely don't listen to the TrueNAS community. Or at least, that's how I read the GP co…

> If you want to use ZFS, then TrueNAS with sensible defaults is a great way to go.

I evaluated TrueNAS Scale about 6 months ago and bailed on it very quickly. My install with their "sensible defaults" ended up having swap partitions on all my spinning disks. That alone was enough for me to decide against using it. One bad choice likely means other bad choices in my experience. They also trampled / reset my 'zfs_zrc_max' tunable every time a VM got started (I think) and it wasn't obvious what was doing it.

I didn't think there was any value in learning the quirks of TrueNAS over doing a setup from scratch, so I ended up going with Ubuntu LTS and doing all my own config / testing.

Re: In OpenZFS and Btrfs, everyone was just guessing

#118

Post by auspiv »

BSDobelix wrote: post #92
RantyDave wrote: post #37

Earlier quoted context omitted.

On the other hand there are a couple of things in ZFS that can make a big difference. For instance: changing the page size to be the same size as Postgresql uses (8kb); disabling the writing of 'last accessed' timestamps; or whether or not the filesystem attempts deduplication. From what I remember (it's been a while) these can be configured on a per-dataset basis, where a dataset is a lightweight filesystem more aki…

For Postgres just follow the original doc: https://openzfs.github.io/openzfs-docs/Performance%20and%20T... compression=lz4 recordsize=32K In postgres -> full_page_writes=off

But also experiment with recordsizes of 8k and 16k. The larger the recordsize, the more potential compression you get, which may be offset by additional delay caused by read/modify/write of the larger record (i.e. reading/writing 32k instead of 8k or 16k).

Re: In OpenZFS and Btrfs, everyone was just guessing

#120

Post by jeffbee »

frankjr wrote: post #45
szpght wrote: post #5

The bug was finally fixed with the following commit: https://github.com/openzfs/zfs/pull/15579/commits/679738cc40... It looks like they managed to exactly pin down what was happening.

A more "correct" fix has been posted https://github.com/openzfs/zfs/pull/15615

Neither of these fixes are great because neither of them include tests. This is a dataloss bug in a filesystem, no existing test covered it, and no new tests were added to demonstrate the efficacy of the supposed fix or prevent regressions.
Post reply on HN