Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TFS: A file system built for performance, space efficiency, and scalability

github.com

Post reply on HN
Print view View original

111–120 of 126 posts

Re: TFS: A file system built for performance, space efficiency, and scalability

#111

Post by tptacek »

dom0 wrote: post #74
fpgaminer wrote: post #66

Earlier quoted context omitted.

Compared to AES, ARX ciphers: 1) Are built from constant time operations, which means they are naturally resistant to side channel attacks (timing, cache, power, etc). 2) Are far simpler in their construction. This makes them easier to reason about and analyze. 3) Related to #2, this also makes them really easy to implement, which means less likelihood of some coding mistake. Beyond that, most recent ARX ciphers also…

ARX constructions are also easier to tune for high software performance, and generally don't require special hardware support, because all CPUs already have fast ARX operations built in.

That's true, but:

* The CPUs that have better-than-ARX (like, fast constant time multiplication) can do better than ARX

* Fast ARX ciphers are still slower than Intel AES hardware.

I like Salsa/ChaCha more than AES, but there's a reason AES is so popular, and it's not incompetence or a conspiracy.

Re: TFS: A file system built for performance, space efficiency, and scalability

#112

Post by tptacek »

ysleepy wrote: post #24

What cipher block mode is used with SPECK? Block based disk encryption is complicated to get right due to replay attacks of blocks over time and IV issues. There are established best-practice compromises with AES, but I dont know if they apply to other block ciphers and doubt they are tested.

XEX, unfortunately. That's a mistake. Unauthenticated tweakable wide-block cipher modes are designed for simulated hardware disk encryption. That's not what an encrypted filesystem is: a filesystem knows where files begin and end, and has space for metadata. Filesystem encryption should use authenticated encryption.

Re: TFS: A file system built for performance, space efficiency, and scalability

#113

Post by ticki_ »

gpm wrote: post #102
ticki_ wrote: post #100

Earlier quoted context omitted.

Author here. > I don't know if the authors are here, but if they are - would you comment on fragmentation and the dangers of growing a filesystem past 95-98% full ? Fragmentation isn't an issue in TFS, at all. Because it is a cluster-based file system. Essentially that means that files aren't stored contagiously, but instead in small chunks. The allocation is done entirely on the basis of unrolled freelists. This doe…

I only have a basic understanding of harddisks/filesystems, but won't that slow down reading/writing on harddisks since the chunks won't be in order and close together?

With modern hard disks, no. They work in sectors.

Re: TFS: A file system built for performance, space efficiency, and scalability

#114

Post by ticki_ »

dap wrote: post #103
ticki_ wrote: post #91

Earlier quoted context omitted.

> I understand this filesystem is still nascent, but shouldn't data integrity at least be one of the design goals? What makes you think it isn't? It definitely is. In fact, it borrows several ideas from ZFS wrt/ integrity. For example, it uses parent block checksums like ZFS.

> What makes you think it isn't? The first section in the README is called "Design goals", with 13 items. None of them is "data integrity", and none of them even talks about validating the data or handling any failures aside from power loss. By contrast, in the canonical slide deck on ZFS[1], the first slide talks about "provable end-to-end data integrity". In the paper[2], "design principles" section 2.6 is "error d…

> The first section in the README is called "Design goals", with 13 items. None of them is "data integrity", and none of them even talks about validating the data or handling any failures aside from power loss.

Fair enough.

Re: TFS: A file system built for performance, space efficiency, and scalability

#115

Post by flukus »

pjmlp wrote: post #110
flukus wrote: post #109

Earlier quoted context omitted.

The more you discourage checking in the harder merging becomes.

In some type of enterprise projects I am quite glad that merge never takes place. I don't want to have a CI build that is green only a few days when all planets are aligned.

> In some type of enterprise projects I am quite glad that merge never takes place.

Surprise! Now you have 2 versions of an app to support, with twice the resource requirements.

Re: TFS: A file system built for performance, space efficiency, and scalability

#116

Post by pjmlp »

flukus wrote: post #115
pjmlp wrote: post #110

Earlier quoted context omitted.

In some type of enterprise projects I am quite glad that merge never takes place. I don't want to have a CI build that is green only a few days when all planets are aligned.

> In some type of enterprise projects I am quite glad that merge never takes place. Surprise! Now you have 2 versions of an app to support, with twice the resource requirements.

No I don't.

I and everyone else on the team enjoys the fact that broken code produced by offshore developers that should have been fired in first place, never touches the official development repository.

Those developers will eventually produce something that passes the unit tests and gets merged, instead of borking the build for weeks.

Should the process done in another form with code reviews and such, yes but that isn't how many enterprise projects are managed.

Note that I am only referring to those that can't learn to code even if we ELI5 them.

There are others on the offshore teams that are highly skilled, but like us onsite, cannot do anything to change the rules of the game.

Re: TFS: A file system built for performance, space efficiency, and scalability

#117

Post by moomin »

pjmlp wrote: post #86
moomin wrote: post #38

Earlier quoted context omitted.

It's more than just a bad VCS. It's a bad bug tracker, a bad build system and a bad workflow system as well. But yeah, if I wanted a set of three letters than suggested quality and reliability, TFS would be near the bottom of my list.

Actually I am yet to find a CI system that allows the TFS workflow of preventing checkins if the unit tests fail. All the other ones I know require creating scripts that will manage branches and merges into the main branch. On TFS I select a check box and go off doing something else.

This thread has got quite long but no-one's mentioned the fact you can achieve the same thing with triggered merges: https://blog.jetbrains.com/teamcity/2013/10/automatic-merge/

Re: TFS: A file system built for performance, space efficiency, and scalability

#118

Post by pjmlp »

moomin wrote: post #117
pjmlp wrote: post #86

Earlier quoted context omitted.

Actually I am yet to find a CI system that allows the TFS workflow of preventing checkins if the unit tests fail. All the other ones I know require creating scripts that will manage branches and merges into the main branch. On TFS I select a check box and go off doing something else.

This thread has got quite long but no-one's mentioned the fact you can achieve the same thing with triggered merges: https://blog.jetbrains.com/teamcity/2013/10/automatic-merge/

Thanks for sharing, I wasn't aware of it.

Re: TFS: A file system built for performance, space efficiency, and scalability

#119

Post by u02sgb »

derefr wrote: post #84
dom0 wrote: post #73

Earlier quoted context omitted.

It's absolutely an impressive feat to pull this off. But it's not quite the same problem as building a robust general-purpose FS for a diverse ecosystem.

True; and yet, it feels like there is much low-hanging fruit left in filesystens that are just built for specific vertically-integrated use-cases. A NAS hardware-appliance company, for example, could likely pull off something similar to what Apple did, and to great benefit.

Or just use btrfs on top of software Linux raid like synology does.

Re: TFS: A file system built for performance, space efficiency, and scalability

#120

Post by hbogert »

moomin wrote: post #83
solatic wrote: post #8

Earlier quoted context omitted.

The readme also alludes to a file system by terminalcloud which is also apparently named TFS, and doesn't explain what the T is supposed to stand for. Really poor choice of name. If they wanted a modular replacement for ZFS then call it MFS or ModFS or something...

If it's a successor, they should follow tradition and just pick the letter after Z.

AAFS it is then. To bad it already exists, and even more confusingly also has something to do with a journal :D
Post reply on HN