Tagsistant: semantic filesystem for Linux • Hacker News Board

Re: Tagsistant: semantic filesystem for Linux

#41

Post by [deleted] » Mon, Jun 12, 2017, 11:09 PM UTC

Pxtl wrote: post #30

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

I remember for a while Google Drive worked on a tags metaphor and it was just too much friction for users. The UI for the traditional file tree is just plain cleaner. Trees provide clear delineations of ownership and categorization. They suffer from the limits of hierarchies, but you can break them down and get a nice tree-view of them, for example. Having spent time categorizing my photos, adding a tag layer on top…

[deleted]

Re: Tagsistant: semantic filesystem for Linux

#42

Post by Risord » Mon, Jun 12, 2017, 11:21 PM UTC

XorNot wrote: post #38

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

The problem with any other model is there are secondary superpowers required to make it work: forget what deleting a file means, what does editing it mean when it can appear in multiple places? Tagging files IMO depends on having a robust deduplication system under the hood.

Does these questions require superpowers? I think no but a bit different thinking model for sure. For example thought that "file is somewhere" is already miss match for model since point of hash based file systems is usually make location irrelevant.

Traditional deletion should be split to real delete and unlink (see: https://news.ycombinator.com/item?id=14541776).

Editing mutable file you just mutate it everywhere since it's just single file after all. Editing immutable content is same as creating new file.

I think system on article doesn't suffer duplication problem (due tagging, for immutability maybe). There is no problem showing file on multiple path or "result of multiple query" without data deduplication unless we somehow try to brute force tag FS over traditional HFS. If I understood your concern correctly...

Re: Tagsistant: semantic filesystem for Linux

#43

Post by Risord » Mon, Jun 12, 2017, 11:37 PM UTC

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

Just in case if you are not aware check RDF/RDFS/OWL. Use cases are quite different but model wise there are similarities (no immutability / hash based tough).

For example case how do you manage details of relationships of tags? Like how do you handle situation where you are accidentally created duplicate tag?

Spoiler: There is not-so-small complexity creep there.

I hope sane conscious trade-offs can be done.

Re: Tagsistant: semantic filesystem for Linux

#44

Post by eriknstr » Mon, Jun 12, 2017, 11:58 PM UTC

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

I too have been thinking about alternatives to hierarchial filesystems. Originally I thought that I wanted a tag based system, but currently I believe that tagging adds too much overhead and that it might not scale well either for the user since the more tags you have the harder they will be to remember and then there is the issue of relationships between tags.

On macOS they have a view that shows you all your files in the order that they were last modified. That has value I believe. Combine that with optional categorization for filtering, so tags but with the tags not being at the very center, and store files by their hash but keep filename for display. Also keeping track of origin of files - user authored vs made by others. Documents with hyperlinks are used for organizing files of current concern. Snapshots of all files are kept so you can delete, replace or update "indexes" over time but still being able to retrieve them later. Append-only except for the pointers that show what indexes are active.

Hope any of that made sense. It's a bit late and besides I didn't want to write "a whole book" in this comment.

Re: Tagsistant: semantic filesystem for Linux

#45

Post by prewett » Tue, Jun 13, 2017, 2:05 AM UTC

I've never been able to figure out what people mean by "tags" and what the benefits are, perhaps someone can enlighten me?

I feel like the only problem with standard hierarchical file systems is that sometimes you want files to show up in multiple places. I think this is only a subset of files, typically "media." A photograph or a song often has multiple categorizations. However, a lot of things, like my tax documents, source code, Word documents, and notes typically only have one place they need to go to. It seems like symlinks or hard links are an already-existing solution; other than lack of great options on managing them, how do tags improve this?

As far as I can tell, tagsistant does have hierarchical tags, so it isn't doing away with hierarchy, which is good. One problem with some tag systems (like Gmail's original) is that under "notes" I have "lectures," "stuff I want to remember," "things like list of books I want to read," "temporary." If I'm looking for photos, I really don't want to see a bunch of notes tags.

Re: Tagsistant: semantic filesystem for Linux

#46

Post by seagreen » Tue, Jun 13, 2017, 2:48 AM UTC

rakoo wrote: post #7

Nice! An equivalent for comparison: https://tmsu.org/

This should be higher up, I think TMSU is the best known product in this space, and a comparison between the two would be useful.

For one thing I think TMSU's achilles heel is handling renaming of files gracefully, I wonder how Tagsistant does it?

Re: Tagsistant: semantic filesystem for Linux

#47

Post by leoc » Tue, Jun 13, 2017, 2:57 AM UTC

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

Well, since you ask, here's Hans Reiser's old stuff:

https://reiser4.wiki.kernel.org/index.php/Future_Vision

https://reiser4.wiki.kernel.org/index.php/V4

(and http://lwn.net/2001/1108/a/reiser4-transaction.php3 )

. And here's some emails etc. I wrote in response:

https://web.archive.org/web/20040728044342/http://www.st-and...

https://www.mail-archive.com/reiserfs-list@namesys.com/msg09...

https://www.mail-archive.com/reiserfs-list@namesys.com/msg20...

, plus some of the discussion threaded from those posts. (Sorry, my stuff needs rewriting and updating but I'm not in the position to do it at present. If there's anything you would like to ask about please do. https://news.ycombinator.com/item?id=9809041 and https://news.ycombinator.com/item?id=10548477 touch on things that are a bit further down the line, but related—in particular, to the handling of "internal metadata" and files with a compound internal structure.)

Re: Tagsistant: semantic filesystem for Linux

#48

Post by jonstewart » Tue, Jun 13, 2017, 4:18 AM UTC

leoc wrote: post #47

nayuki wrote: post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

Well, since you ask, here's Hans Reiser's old stuff: https://reiser4.wiki.kernel.org/index.php/Future_Vision https://reiser4.wiki.kernel.org/index.php/V4 (and http://lwn.net/2001/1108/a/reiser4-transaction.php3 ) . And here's some emails etc. I wrote in response: https://web.archive.org/web/20040728044342/http://www.st-and... https://www.mail-archive.com/reiserfs-list@namesys.com/msg09... https://www.mail-archive.com…

Yes, the first thing that sprang to my mind was Reiser's design documents from 15+ years ago. Pity...

Re: Tagsistant: semantic filesystem for Linux

#49

Post by tyingq » Tue, Jun 13, 2017, 4:22 AM UTC

kwhitefoot wrote: post #16

someSven wrote: post #10

I often wanted a tag system, but this is not what I was picturing, I think. I'll rather like to have a database and an entry in the context menue to tag files without moving them. It should also be able to identify the file if is being moved or renamed.

It must be possible to get the same effect without copying the files by using links. If it is all on the same file system then hardlinks would work and be very space efficient.

Extended attributes can do this for most unixy operating systems. See fsetattr and fgetattr. It's a bit tricky in that the standard tools, like find, don't support them directly. But, the tags stay with the files, no separate DB needed. You do have to pay attention to copying, backups, etc, to make sure they are preserved, and no separate DB means all "queries" are full scans.

Re: Tagsistant: semantic filesystem for Linux

#50

Post by rakoo » Tue, Jun 13, 2017, 7:39 AM UTC

yourapostasy wrote: post #29

clumsysmurf wrote: post #22

Earlier quoted context omitted.

The article mentions a few technologies like IPFS, so I'm just mentioning Camlistore for completeness because its in this space too. https://camlistore.org/ I'm reading "The Science of Managing Our Digital Stuff (MIT Press)". They seem to prefer hierarchies (I'm not far through the book yet). https://www.amazon.com/Science-Managing-Digital-Stuff-Press/...

> They seem to prefer hierarchies... We might not be able to get away from hierarchies, if experience with "memory palaces" [1] gives an indication of how most of us remember (would be interesting if we could identify the memory palace equivalents for those who remember using auditory, visual, and tactile forms [2]). Most memory palaces' dendritic structure bears a striking resemblance to a hierarchical system. I sti…

I second the suggestion to give a look at camlistore. It's designed to be a content storage for everything with automatic features extraction; the first use case implemented by the authors is to extract all the metadata from photos as soon as they are uploaded, so you can search for, say, panoramas in Lisbon in the last 3 months. Other document types are already implemented and you can have your own depending on your use case. Coming from a Googler, it's no surprise that the primary interface is "tag stuff, and use search"