Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZFS: Apple's new filesystem that wasn't (2016)

ahl.dtrace.org

Post reply on HN
Print view View original

191–200 of 209 posts

Re: ZFS: Apple's new filesystem that wasn't (2016)

#191

Post by GeekyBear »

Marsymars wrote: post #187
GeekyBear wrote: post #171

Earlier quoted context omitted.

Back in 2011, Apple paid $500 million for Anobit, a company producing enterprise grade SSD controllers. Since Apple was already integrating custom SSD controllers onto their A series SOCs, presumably the purchase was about Anobit patents. > Anobit appears to be applying a lot of signal processing techniques in addition to ECC to address the issue of NAND reliability and data retention... promising significant improve…

That's excellent if you're using Apple SSD storage, but seems unhelpful if you're running, say, a large array of HDDs connected to your Mac.

Sure, if you are managing a large amount of external data, you may want to go with a RAID array or the existing OpenZFS port, but even Windows 11 still defaults to NTFS, which offers less resilience.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#192

Post by runjake »

kjkjadksj wrote: post #183
GeekyBear wrote: post #171

Earlier quoted context omitted.

Back in 2011, Apple paid $500 million for Anobit, a company producing enterprise grade SSD controllers. Since Apple was already integrating custom SSD controllers onto their A series SOCs, presumably the purchase was about Anobit patents. > Anobit appears to be applying a lot of signal processing techniques in addition to ECC to address the issue of NAND reliability and data retention... promising significant improve…

What is the devices lifetime in apples eyes? 5 years till support is dropped? Gulp.

No, generally much longer.

Apple products go vintage after they've stopped selling something for 5 years. Obsolete after 7 years.

https://support.apple.com/en-us/102772

Re: ZFS: Apple's new filesystem that wasn't (2016)

#193

Post by Dwedit »

ryao wrote: post #76
whartung wrote: post #63

As a desktop user, I am content with APFS. The only feature from ZFS that I would like, is the corruption detection. I honestly don't know how robust the image and video formats are to bit corruption. On the one hand, potentially, "very" robust. But on the other, I would think that there are some very special bits that if toggled can potentially "ruin" the entire file. But I don't know. However, I can say, every time…

> I honestly don't know how robust the image and video formats are to bit corruption. It depends on the format. A BMP image format would limit the damage to 1 pixel, while a JPEG could propagate the damage to potentially the entire image. There is an example of a bitflip damaging a picture here: https://arstechnica.com/information-technology/2014/01/bitro... That single bit flip ruined about half of the image. As for…

If you know that a particular JPEG block is corrupted, it could be possible to isolate the damage to just that block. You know the next block should not have a hard edge against the block above it, so you could try to find something that minimizes the vertical edge strength for the remaining blocks in the row.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#194

Post by 3np »

mdaniel wrote: post #77
whartung wrote: post #63

As a desktop user, I am content with APFS. The only feature from ZFS that I would like, is the corruption detection. I honestly don't know how robust the image and video formats are to bit corruption. On the one hand, potentially, "very" robust. But on the other, I would think that there are some very special bits that if toggled can potentially "ruin" the entire file. But I don't know. However, I can say, every time…

> Just trying to set it up on a single USB drive That's the fault of macOS, I also experienced 100% CPU and load off the charts and it was kernel_task jammed up by USB. Once I used a Thunderbolt enclosure it started to be sane. This experience was the same across multiple non-Apple filesystems as I was trying a bunch to see which one was the best at cross-os compatibility Also, separately, ZFS says "don't run ZFS on…

I haven't had any issues with attaching normal SATA drives with SATA-over-USB cables. Meanwhile some otherwise apparently fine USB thumb drives got abysmal results (ZFS on Linux.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#195

Post by aaronmdjones »

thfuran wrote: post #34
yjftsjthsd-h wrote: post #29

Earlier quoted context omitted.

There is a rule of thumb that you should have at least 1 GB of RAM per TB of disk when using deduplication . That's.... Different.

So you've never seen the people saying you should steer clear of ZFS unless you're going to have an enormous ARC even when talking about personal media servers?

I have seen people saying that yeah. I've also completely ignored them.

I have a 64TB ZFS pool at home (12x8TB drives in an 11w1s RAID-Z3) on a personal media server. The machine has been up for months. It's using 3 GiB of RAM (including the ARC) out of the 32 I put in it.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#196

Post by redeeman »

ryao wrote: post #73
ahl wrote: post #26

Earlier quoted context omitted.

It's very amusing that this kind of legend has persisted! ZFS is notorious for *noticing* when bits flip, something APFS designers claimed was rare given the robustness of Apple hardware.[1][2] What would ZFS on iPhone have looked like? Hard to know, and that certainly wasn't the design center. Neither here nor there, but DTrace was ported to iPhone--it was shown to me in hushed tones in the back of an auditorium onc…

I did early ZFSOnLinux development on hardware that did not have ECC memory. I once had a situation where a bit flip happened in the ARC buffer for libpython.so and all python software started crashing. Initially, I thought I had hit some sort of blizzard bug in ZFS, so I started debugging. At that time, opening a ZFS snapshot would fetch a duplicate from disk into a redundant ARC buffer, so while debugging, I ran cm…

erhm, isnt zfs supposed to store the checksum of records stored in the arc, and verify on read? are you sure this is what happened?

Re: ZFS: Apple's new filesystem that wasn't (2016)

#197

Post by CTDOCodebases »

Marsymars wrote: post #189
water9 wrote: post #147

Earlier quoted context omitted.

The checksum doesn’t help you fix the flipped bit nor does it tell you which bit flipped. You would have to re-create from a complete back up instead of using the efficiency of parity discs. Basically Raid 1 vs Raid 5

There are tools that give you hashing + parity, e.g. SnapRAID: https://www.snapraid.it/

+1 For Snapraid.

If you already have some data on ext4 disk(s) and don't want to deal with the the issues of using ZFS/BTRFS then it's a no brainer. Dynamically resizing the "array" is super simple and it works really well with MergerFS.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#198

Post by water9 »

rollcat wrote: post #163
water9 wrote: post #147

Earlier quoted context omitted.

The checksum doesn’t help you fix the flipped bit nor does it tell you which bit flipped. You would have to re-create from a complete back up instead of using the efficiency of parity discs. Basically Raid 1 vs Raid 5

I wonder how hard would it be to detect which single bit was flipped? As ryao noted, in JPEGs it's immediately obvious where the image was corrupted - by visual inspection. Similar for videos, you only need to inspect the data following a single I-frame. Even for bitmap/text files, you could just scan the entire file, try flipping one bit at a time, and compare the result with the checksum. Unlike e.g. KDFs, checksum…

What if you had two bit flips?

Re: ZFS: Apple's new filesystem that wasn't (2016)

#199

Post by worthless-trash »

wkat4242 wrote: post #137
worthless-trash wrote: post #129

Earlier quoted context omitted.

There is an increase of posts with casual confidence in their own absolute correctnes. I originally attributed it to the influence of llms, but it is becoming so common now it is hard to dismiss.

I never claimed absolute correctness, just stating what I see with the other BSD users I know.

You did say that a most linux admins dont really care.. which has been edited out now.

Re: ZFS: Apple's new filesystem that wasn't (2016)

#200

Post by wkat4242 »

worthless-trash wrote: post #199
wkat4242 wrote: post #137

Earlier quoted context omitted.

I never claimed absolute correctness, just stating what I see with the other BSD users I know.

You did say that a most linux admins dont really care.. which has been edited out now.

Huh no I never spoke of Linux users
Post reply on HN