Live data from Hacker News

TFS: A file system built for performance, space efficiency, and scalability

github.com

61–70 of 126 posts

Re: TFS: A file system built for performance, space efficiency, and scalability

#61
post #40

While there are many good arguments to be made against AES in favor of ARX construction ciphers, the choice of SPECK for this is not okay. The correct choice of an ARX cipher would have been something like ChaCha20 or Salsa20.

ChaCha2 isn't a block cipher. It's a stream cipher, and would thus require storage of IVs ... which, well, is going to expensive space-wise.

Re: TFS: A file system built for performance, space efficiency, and scalability

#62

Good luck. There is long road ahead. Both ZFS and Btrfs were initially developed by really high caliber people and experts with good track record. ZFS had five years for full time development until release, next five years to get close to the features and stability that ZFS has now. Btrfs started 10 years ago and it's still trying to catch up.

Has anyone done an analysis of the number of logic bugs in ZFS and btrfs vs. memory safety or concurrent updates to memory, etc.? Also, I'll point out that Apple just dropped a new FS on millions of devices, with no issues... that was developed in 3 or 4 years. I'm still blown away that they pulled that off.

They have full control of that environment though. So they tested it on a few hundred devices, and by that exhausted all the possible configurations. And when all worked, they knew they have a pretty good indicator for successful deployment on all those millions of devices.

Re: TFS: A file system built for performance, space efficiency, and scalability

#63
post #51

looks like a one man/woman show. the graphs show most of the activity only by one contributor. :(

I don't consider that a bad thing. The project may pick up more contributors down the road — or it might not. Either way, that doesn't speak negatively of the project itself.

Re: TFS: A file system built for performance, space efficiency, and scalability

#64
post #22

I wonder would it be possible to use it somehow with Linux (in kernel space, not with fuse, because fuse has work slower because of necessary context switches from kernel to user space). I mean it is interesting can a wrapper kernel module be written for interfacing with Rust code, or there are some obstacles that would prevent from doing it efficiently.

https://github.com/tsgates/rust.ko#kernel-api-and-abi-stabil...

The unstable Kernel ABI is really interesting. It's one of the reasons we see all these shims between proprietary drivers and the kernel (nvidia/amd), that and licensing of course.

Re: TFS: A file system built for performance, space efficiency, and scalability

#65
post #59
post #57

Earlier quoted context omitted.

And they probably won't for political reasons. Salsa20 was created by DJB who has previously fought with US Gov against encryption export ban and also they have their own suits they prefer you to use (for their own reasons). Here is a reason why you can't trust US Gov for creating secure encryption: https://en.wikipedia.org/wiki/Dual_EC_DRBG

It doesn't matter if I trust or distrust the US government for creating secure encryption. There are laws in place across a plethora of industries REQUIRING FIPS140-2. If you want to do business in the US, it's a basic requirement. At the end of the day the odds of this filesystem gaining any traction are basically 0. I'm just pointing out that not supporting an encryption algorithm that can meet the FIPS requirement…

> At the end of the day the odds of this filesystem gaining any traction are basically 0. I'm just pointing out that not supporting an encryption algorithm that can meet the FIPS requirement makes it basically a non-starter for any commercial application.

In the US. In many other countries, it might gain lots of traction.

Re: TFS: A file system built for performance, space efficiency, and scalability

#66
post #52
post #40

While there are many good arguments to be made against AES in favor of ARX construction ciphers, the choice of SPECK for this is not okay. The correct choice of an ARX cipher would have been something like ChaCha20 or Salsa20.

At the risk of discussion hijack, what are these arguments? Any links? (I mean on ARX generally. Agree about Speck.)

Compared to AES, ARX ciphers:

1) Are built from constant time operations, which means they are naturally resistant to side channel attacks (timing, cache, power, etc).

2) Are far simpler in their construction. This makes them easier to reason about and analyze.

3) Related to #2, this also makes them really easy to implement, which means less likelihood of some coding mistake.

Beyond that, most recent ARX ciphers also have a few other advantages over AES. For example, Threefish has a built-in tweak field, which makes using it infinitely easier in practice.

EDIT: In case you're hungry for more detailed explanations, I highly recommend reading the papers for Salsa/Chacha and Threefish. They're very well written, easy to understand even if you don't have a lot of experience with cryptography, and they have sections that explain the design decisions in enlightening detail.

Re: TFS: A file system built for performance, space efficiency, and scalability

#67

Good luck. There is long road ahead. Both ZFS and Btrfs were initially developed by really high caliber people and experts with good track record. ZFS had five years for full time development until release, next five years to get close to the features and stability that ZFS has now. Btrfs started 10 years ago and it's still trying to catch up.

I think you think to highly of the btrfs group. There not bad people but Filesystems are HARD to get right and they simply are not in the same league as the zfs developers were. The well known raid 5 issue btrfs had shows a prime example that the btrfs simply is not in the same class as zfs' design.

I wouldn't compare the two.

Re: TFS: A file system built for performance, space efficiency, and scalability

#68
I searched the github page and this HN comment thread for the string "frag" and got nothing ...

I don't know if the authors are here, but if they are - would you comment on fragmentation and the dangers of growing a filesystem past 95-98% full ?

In the world of ZFS, performance can become significantly degraded with as low as 90% space filled. Further, our experience has been that you can permanently degrade filesystem performance by churning the usage above 95% for any significant amount of time. Which is to say, even if you reduce usage back down to 80%, the zpool maintains poor performance until it is destroyed and recreated.

This is exactly what you would expect to see with a fragmenting filesystem that has no defrag tool.

Unfortunately, creating a defrag tool for ZFS is a very daunting technical hurdle and it appears that nobody is interested in pursuing it.

How does TFS behave ? Does it have, or do you plan for it to have, a defrag utility ?

Re: TFS: A file system built for performance, space efficiency, and scalability

#69
post #62

Earlier quoted context omitted.

Has anyone done an analysis of the number of logic bugs in ZFS and btrfs vs. memory safety or concurrent updates to memory, etc.? Also, I'll point out that Apple just dropped a new FS on millions of devices, with no issues... that was developed in 3 or 4 years. I'm still blown away that they pulled that off.

They have full control of that environment though. So they tested it on a few hundred devices, and by that exhausted all the possible configurations. And when all worked, they knew they have a pretty good indicator for successful deployment on all those millions of devices.

It's also used only with their specific flash storage devices.

Re: TFS: A file system built for performance, space efficiency, and scalability

#70
post #62

Earlier quoted context omitted.

Has anyone done an analysis of the number of logic bugs in ZFS and btrfs vs. memory safety or concurrent updates to memory, etc.? Also, I'll point out that Apple just dropped a new FS on millions of devices, with no issues... that was developed in 3 or 4 years. I'm still blown away that they pulled that off.

They have full control of that environment though. So they tested it on a few hundred devices, and by that exhausted all the possible configurations. And when all worked, they knew they have a pretty good indicator for successful deployment on all those millions of devices.

Why downplay it?

It's an impressive feat, regardless of the differences in target devices. Even with the hardware configurations well known, the fact that it was done at such a large scale successfully means that even unusual edge conditions didn't crop up.

This shouldn't be downplayed, it actually speaks to why it's important to have incremental stages of software delivery. First target highly constrained environments (iOS, Watch OS, tvOS), then work on the more difficult and less constrained general computing environment.

Post reply on HN