Live data from Hacker News

TFS: A file system built for performance, space efficiency, and scalability

github.com

21–30 of 126 posts

Re: TFS: A file system built for performance, space efficiency, and scalability

#21

>The system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won't ever damage the system. What is the difference here between a hardware failure and a unexpected power failure?

Unexpected shutdown would just use the features of a Journaling file system to recover the data. A hardware failure would be a Raid controller failure or malfunction or a hard drive / SSD failure .

Re: TFS: A file system built for performance, space efficiency, and scalability

#22

I wonder would it be possible to use it somehow with Linux (in kernel space, not with fuse, because fuse has work slower because of necessary context switches from kernel to user space). I mean it is interesting can a wrapper kernel module be written for interfacing with Rust code, or there are some obstacles that would prevent from doing it efficiently.

https://github.com/tsgates/rust.ko#kernel-api-and-abi-stabil...

Re: TFS: A file system built for performance, space efficiency, and scalability

#23

>The system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won't ever damage the system. What is the difference here between a hardware failure and a unexpected power failure?

The section "2. Hardware Assumptions" in the Sqlite docs here has some interesting related stuff: https://www.sqlite.org/atomiccommit.html

Re: TFS: A file system built for performance, space efficiency, and scalability

#24
What cipher block mode is used with SPECK? Block based disk encryption is complicated to get right due to replay attacks of blocks over time and IV issues. There are established best-practice compromises with AES, but I dont know if they apply to other block ciphers and doubt they are tested.

Re: TFS: A file system built for performance, space efficiency, and scalability

#25
post #9

Is the choice of SPECK a serious decision or is it some kinda of a political/satire/parody thing?

I'm fairly sure it's serious, though if you look at the specification it's made to be easily swapable. It's specified by a 'vdev', which is a layer on top of the core filesystem specified by a 16bit int. You could easily make another for ChaCha20 and use that instead (or also).

Re: TFS: A file system built for performance, space efficiency, and scalability

#27
post #17
post #4

"Team Foundation Server" (Microsoft's version control server) is what goes through my mind... poor choice of naming...

So you have to add “fs” or “filesystem“ to your search queries, or not even that if the rest of the query gives sufficient context. Everyone makes name conflicts in independent domains to be a much bigger problem than they are in reality. There’s Amazon rainforest, Amazon the ecommerce website, and Amazon the cloud company. How often do you have any problem differentiating between them?

Well.. let's google it... "TFS" .. hmm, abbridged lead in to the wikipedia article at the top... And though I'm not a systems programmer likely to implement, or support the code for a filesystem, I am A programmer, and work in IT... And systems operators are also likely to come accross TFS (Team Foundation Server) in terms of supporting a deployment of it.

Though, they've started to refer to the source control protocol implementation as TFVC, since TFS supports git as well now. It does seem to have some conflicts, and even notes another file system called TFS themselves.

In this case, I'm pretty sure another name might be a better idea. Hell, TFS the version control system and the other file system are more well known than Firebird the database when Mozilla renamed their shiny new browser.

Re: TFS: A file system built for performance, space efficiency, and scalability

#28

>The system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won't ever damage the system. What is the difference here between a hardware failure and a unexpected power failure?

I think the spec answers your question here

  TFS provides following guarantees:
  
   - Unless data corruption happens, the disk should never be 
     an inconsistent state \footnote{TFS achieves this without using
     journaling or a transactional model.}. Poweroff and the alike
     should not affect the system such that it enters an invalid or
     inconsistent state.
    
  Provided that following premises hold:

   - Any sector (assumed to be a power of two of at least
     \minimumsectorsize bytes) can be read and written atomically,
     i.e. it is never partially written, and interrupting the write
     will never render the disk in a state in which it not already
     is written or retaining the old data.
  
  Data corruption can break these guarantees or premises, and TFS
  encompasses certain measures against such corruption, but they are
  strictly speaking heuristic, like any error detection and correction
  method, as the damage could be across all the disks.

Re: TFS: A file system built for performance, space efficiency, and scalability

#30

>The system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won't ever damage the system. What is the difference here between a hardware failure and a unexpected power failure?

A power-off is a simple fault, with fairly well-defined effects. It's actually one of the easiest cases for a data-storage system to deal with. "Hardware failure" includes all manner of crazy Byzantine faults, many of which are literally impossible to deal with. What this is saying is that TFS's model of the underlying hardware is that it will either execute all writes correctly or stop executing any.

Sadly, a lot of hardware has much more complicated behavior than that. A lot of RAID cards in particular will lie about what was actually written, so in a power-off scenario later writes might have made it while earlier ones didn't, writes can be incomplete, etc. I don't mean this as a knock against TFS. It's more of a suggestion that the fault model be expanded to include at least a few more possibilities.

Post reply on HN