Live data from Hacker News

Tagsistant: semantic filesystem for Linux

tagsistant.net

21–30 of 71 posts

Re: Tagsistant: semantic filesystem for Linux

#21
post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

I imagine you are familiar with BFS (https://en.m.wikipedia.org/wiki/Be_File_System) already, but if you aren't I'm sure you will find it interesting.

Re: Tagsistant: semantic filesystem for Linux

#22
post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

The article mentions a few technologies like IPFS, so I'm just mentioning Camlistore for completeness because its in this space too.

https://camlistore.org/

I'm reading "The Science of Managing Our Digital Stuff (MIT Press)". They seem to prefer hierarchies (I'm not far through the book yet).

https://www.amazon.com/Science-Managing-Digital-Stuff-Press/...

Re: Tagsistant: semantic filesystem for Linux

#23
I've been thinking increasingly of something along these lines myself, with Tagsistant being among the systems which have come up in my own research.

The problems, variously, are that fixed-name hierarchical-storage filesystems meet the needs of document-based storage, projects, workflows, sharing, and lifecycle exceedingly poorly.

The problem is coming up with a better option.

A filesystem-based approach has the advantage that it's low-level, not tied in to a single application or toolsuite, and may be extensible.

Among the questions I've turned up include identifying what specific problems this is trying to solve, distinguishing between public and private information, and what levels of standardisation might apply. There are also some very significant questions about privacy and data leakage.

My current thoughts are largely grouped around a documents-based system (provisionally, "/docfs"), and an online or Web-oriented system (provisionally, "/webfs"), both under an umbrella context system, KFC (for KFC's Fine Context). Mostly considering the domain space, workflows, and possible solution-shaped objects. Part of that (largely focused on Web access) discussed here:

https://www.reddit.com/r/dredmorbius/comments/6bgowu/what_if...

Re: Tagsistant: semantic filesystem for Linux

#24
post #13

Humans have to tag the files, though. This is the same problem which kept the "semantic web" from going anywhere.

Humans already have to pick a path and name for the file in a hierarchical filesystem. I don't see how this is any more difficult, and it might even be easier.

Re: Tagsistant: semantic filesystem for Linux

#25

Great project! I feel that the problem of archiving files is not well served by the POSIX file system, and deserves attention. Gaps are in data safety and backup capabilities (dropbox is a huge leap forward here) and document retrival. Usually external idices (which go out of sync) are used to query file names. Also there is no way to attach semantic metadata to files (appart from date stamps, and permissions). This…

FUSE has extended (and extensible) file attributes, which seem of interest.

https://en.m.wikipedia.org/wiki/Extended_file_attributes

Re: Tagsistant: semantic filesystem for Linux

#26
post #13

Humans have to tag the files, though. This is the same problem which kept the "semantic web" from going anywhere.

That's where I see a possible out.

There are extensive collections of works which are systematically categorised. We call themm "libraries". One in particular, the U.S. Library of Congress has a corpus of 24 million catalogued works, using an open categorisation scheme, which 1) can be used to apply classifications to extant works and 2) serve as a training set by which unclassified works might be classified.

Distinguishing between expert, automatic, and local classification of content might be useful.

There are other copora, including scientific and commercially-published article indices (of varying degrees of public accessiblity), and more.

It turns out that the process of classifying information has been going on for a while.

Of possibly related interest:

http://historyofinformation.com

(The bits applying to encyclpaedists and library catalog classifications are of particular interest here.)

Re: Tagsistant: semantic filesystem for Linux

#27
post #13

Humans have to tag the files, though. This is the same problem which kept the "semantic web" from going anywhere.

Humans have to tag the files, though.

Only once, if done properly. Look at music tagging. A good system ought to have canonical tags for everything.

User-created files could also have a lot of auto-generated tags too. I'm thinking along the lines of email address/URL origin, Exif metadata, source code tags (ctags/etags), keyword extraction from prose text (via machine learning models)...

Beyond all that, though, would be your standard date and timestamps, your document name (which could be non-unique), and a project-based tagging scheme.

Take a look at the Library of Congress's MARC project [0][1]. It's the most ambitious tagging project I'm aware of.

[0] https://www.loc.gov/marc/

[1] https://en.wikipedia.org/wiki/MARC_standards

Re: Tagsistant: semantic filesystem for Linux

#28
post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

Hierarchical file systems lack expressiveness and are awkward in places.

Hierarchical file systems made the most sense back in the days of the spatial desktop metaphor [0] pre-OS X classic Mac OS Finder. The ability to organize your files in a spatial manner and have the system preserve the one-to-one relationship between a file and its (virtual) physical location within the system was what made it work.

As soon as the browser metaphor (or navigational) file manager [1] took over (owing much of its success to the web browser), this relationship was lost and the system became unwieldy.

[0] https://arstechnica.com/apple/2003/04/finder/3/

[1] https://en.wikipedia.org/wiki/File_manager#Navigational_file...

Re: Tagsistant: semantic filesystem for Linux

#29
post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

The article mentions a few technologies like IPFS, so I'm just mentioning Camlistore for completeness because its in this space too. https://camlistore.org/ I'm reading "The Science of Managing Our Digital Stuff (MIT Press)". They seem to prefer hierarchies (I'm not far through the book yet). https://www.amazon.com/Science-Managing-Digital-Stuff-Press/...

> They seem to prefer hierarchies...

We might not be able to get away from hierarchies, if experience with "memory palaces" [1] gives an indication of how most of us remember (would be interesting if we could identify the memory palace equivalents for those who remember using auditory, visual, and tactile forms [2]). Most memory palaces' dendritic structure bears a striking resemblance to a hierarchical system.

I still find an overall hierarchical structure, combined with indexing and tagging, as the most flexible system with today's technology. I'm looking for ways to implement automated tagging using auto-summarization, voice commands, and automatic environment-contextual cue gathering, and expanding the indexing power with automated ontology extraction. Primitive example: I pick up an incoming call from a client, the system automatically transcribes the conversation, identifies the client, files the recording and transcription to a project folder, analyzes the content of the discussion, and auto-links relevant emails, chats and documents with bi-directional hyperlinks based upon concepts vocalized and conceptual relationship maps extracted based upon a crude initial morphological analysis of the conversation. The accuracy doesn't need to be astounding for this to have use to me; just a crude approximation is sufficient for me to start with.

[1] https://en.wikipedia.org/wiki/Method_of_loci

[2] https://en.wikipedia.org/wiki/Storage_(memory)#Short-term_me...

Re: Tagsistant: semantic filesystem for Linux

#30
post #14

Hierarchical file systems lack expressiveness and are awkward in places. In my day-to-day computing this has become more apparent and problematic with each passing year. For example, I dislike that you are forced to give unique names for each file, that you can classify each file in only one way, and that you can't tell how many copies you own of a piece of data. The smallest step up from a hierarchical file system i…

I remember for a while Google Drive worked on a tags metaphor and it was just too much friction for users. The UI for the traditional file tree is just plain cleaner. Trees provide clear delineations of ownership and categorization. They suffer from the limits of hierarchies, but you can break them down and get a nice tree-view of them, for example.

Having spent time categorizing my photos, adding a tag layer on top of the existing filesystem is a great idea, but using it as a replacement for the traditional directory structure isn't.

Simple operations like "delete everything in here" becomes complicated under a tag structure now that we no longer have a strict concept of A is within B.

Post reply on HN