Earlier quoted context omitted.
Deduplication in ZFS can be pushed onto dedicated flash storage now instead of requiring gobs of RAM.
Great channel for side attacks btw.
Without having physical access to my hardware, where every disk is FDE anyway?
In OpenZFS and Btrfs, everyone was just guessing
141–150 of 172 posts
Re: In OpenZFS and Btrfs, everyone was just guessing
#142Earlier quoted context omitted.
A lot of end users run ZFS through TrueNAS/FreeNAS and I indeed found that community to spread a lot of misinfo and un-actionable advice. I liken them to gamers lapping up any snake oil solution to get 0.1 fps more. I would suggest anyone trying to learn about ZFS to avoid it, it definitely was counter-productive to me. Reddit, random blogs and official documentation (often that by Oracle) turned out to be much bette…
Reminds me of a game (MMO) I play on the weekends. Any time someone new joins, they always ask how come frame rates are so low. Then we begin the ”set your graphic settings to high” conversation. They immediately say we are full of shit. It’s already set to low and getting 10 fps. Eventually, someone explains: “just try it, or stop complaining.” 30s later, they exclaim: “holy ** I’m getting 60 fps now!” Sometimes, wh…
Wow! Thats a great story
Re: In OpenZFS and Btrfs, everyone was just guessing
#143I don't know about BTRFS, but with ZFS there's a lot of bad information and assumptions floating around. It's one of my favorite topics to ask AI about because it shows the obvious deficiencies in AI as it regurgitates all the bad info. So when you go to the issue tracker, it seems plausible to me that you're going to find issues where the devs can't explain what's happening because the user(s) might be doing somethi…
Tbh guessing is part of the way to solve a bug. You need to make a hypothesis and the test if it is true. If not try another.
Saying the zfs/btrfs teams have no idea means the person has no idea about software development process ... It is a system none fully knows a system that complex.
If they did, the bug wouldn't have happened.
Re: In OpenZFS and Btrfs, everyone was just guessing
#144I don't know about BTRFS, but with ZFS there's a lot of bad information and assumptions floating around. It's one of my favorite topics to ask AI about because it shows the obvious deficiencies in AI as it regurgitates all the bad info. So when you go to the issue tracker, it seems plausible to me that you're going to find issues where the devs can't explain what's happening because the user(s) might be doing somethi…
On the other hand there are a couple of things in ZFS that can make a big difference. For instance: changing the page size to be the same size as Postgresql uses (8kb); disabling the writing of 'last accessed' timestamps; or whether or not the filesystem attempts deduplication. From what I remember (it's been a while) these can be configured on a per-dataset basis, where a dataset is a lightweight filesystem more aki…
> disabling the writing of 'last accessed' timestamps
atime is the most useless filesystem feature I’ve heard of. But on most Unix-like systems it’s enabled by default and the main thing it does is turn reads into writes, degrading performance. All the while ctime isn’t a standard Unix filesystem feature. And ctime to me always seemed infinitely more useful (and cheaper!) than atime.
Re: In OpenZFS and Btrfs, everyone was just guessing
#145Earlier quoted context omitted.
What you are describing sounds like the usual backup strategies. Filesystem bugs that silently corrupt your data will also get synced and backed up.
> Filesystem bugs that silently corrupt your data will also get synced and backed up This is a very easy problem to solve: don't do incremental backups. Or have N backups and rotate, which isn't as good but still gives you more time to notice. Hard drives are cheap.
Still doesn't excuse a filesystem claimed to be designed for reliability from having shoddy development practices.
Re: In OpenZFS and Btrfs, everyone was just guessing
#146Earlier quoted context omitted.
So user's are at fault for trusting a filesystem that advertises itself as being reliable? Even if user's should use multiple redundant filesystems (lol ok), that doesn't invalidate the criticisms in the linked forum post.
> So user's are at fault for trusting a filesystem that advertises itself as being reliable? Yes, absolutely 100% at fault. There are many failure modes here beyond corruption: human error, electrical surges, flooding, fire. Hell, somebody could break into your house and steal the NAS! Filesystem bugs aren't a big deal. If they are a big deal to you, it's because you're trusting a system with a single point of failur…
Could you imagine if we treated other pieces of technology the same way backup fanatics talk about backups?
It's your fault for not having 2 extra phones on you in case your phone bricks itself at a bad time.
It's your fault for not encrypting your signal messages by hand, encryption bugs aren't a big deal, if they're a big deal to you, it's because you're trusting a system with a single point of failure, and the consequences of that inevitable failure are not acceptable to you. Stop doing that.
Your position sounds insane to me of applied to any other piece of technology
Re: In OpenZFS and Btrfs, everyone was just guessing
#147Earlier quoted context omitted.
Deduplication requires an absolutely enormous amount of memory, unless you have a dataset that has a huge amount of duplication for some reason (maybe a company or school where a lot of people have their own network storage and put the same large files in?) it is usually never the right choice to enable dedup. It is cheaper to buy more disk than the memory to enable dedup (somebody could run the numbers). ZFS ARC als…
Deduplication in ZFS can be pushed onto dedicated flash storage now instead of requiring gobs of RAM.
How to do this?
Re: In OpenZFS and Btrfs, everyone was just guessing
#148Earlier quoted context omitted.
On the other hand there are a couple of things in ZFS that can make a big difference. For instance: changing the page size to be the same size as Postgresql uses (8kb); disabling the writing of 'last accessed' timestamps; or whether or not the filesystem attempts deduplication. From what I remember (it's been a while) these can be configured on a per-dataset basis, where a dataset is a lightweight filesystem more aki…
Also, compression. CPUs are more than capable of handling it on the fly, and the reduced dataset size helps IO. This [0] is a brief post on MySQL, but most of it applies equally to Postgres. [0]: https://www.percona.com/blog/mysql-zfs-performance-update/
I'd be curious if the addition of zstd compression has any effect on the results. Could be a fun follow-up follow-up :-)
Re: In OpenZFS and Btrfs, everyone was just guessing
#149Earlier quoted context omitted.
> So user's are at fault for trusting a filesystem that advertises itself as being reliable? Yes, absolutely 100% at fault. There are many failure modes here beyond corruption: human error, electrical surges, flooding, fire. Hell, somebody could break into your house and steal the NAS! Filesystem bugs aren't a big deal. If they are a big deal to you, it's because you're trusting a system with a single point of failur…
Could you imagine if we treated other pieces of technology the same way backup fanatics talk about backups? It's your fault for not having 2 extra phones on you in case your phone bricks itself at a bad time. It's your fault for not encrypting your signal messages by hand, encryption bugs aren't a big deal, if they're a big deal to you, it's because you're trusting a system with a single point of failure, and the con…
Equating a filesystem corruption with bricking a device shows you don't get it. Filesystems are ephemeral: all I have to do after a corruption is reinstall the system, and it's as good as new.
You also missed the most important part of what I said:
>> because you're trusting a system with a single point of failure, and the consequences of that inevitable failure are not acceptable to you.
I don't care if my phone fails: it's disposable. Sure, there are times it might be inconvenient, but it's not like I need it to drive my car or get into my house.
If your phone failing is some huge big horrible problem for you, yeah, you absolutely ought to carry two phones around. It's unreasonable to expect consumer grade technology to be that reliable.
Re: In OpenZFS and Btrfs, everyone was just guessing
#150In many ways, ZFS is an ideal case. It was started at Sun, where robust and thoughtful engineering was the norm. That gives us an extraordinarily well–built foundation to build on. We lost a lot when Sun was killed off by Oracle.