I have thought about this a bit and I think if similarity hashes (probably LSH forest) were used, automatic tagging could occur with a preset of hashes.
Designing better file organization around tags, not hierarchies (2017)
101–110 of 169 posts
Re: Designing better file organization around tags, not hierarchies (2017)
#102[deleted]
Re: Designing better file organization around tags, not hierarchies (2017)
#103Have you considered a file system organized as a timeline that _also_ supports tagging? I find one of the key concepts that's not a first-class concept is _when_ the file was modified. Rather than a file-and-folder physical analogy for the file system UI, I think a timeline-oriented UI could present some advantages for the way that humans actually think and work. Tags would be a helpful orthogonal organization scheme…
I have thought about file tagging for over a decade, before setting out to write the article. But a timeline-oriented file organization only came to my awareness near the end of writing the first draft. A year has passed since I wrote the article, and the idea of timeline presentation has grown on me a lot. Especially because I use numerous data systems daily that are already time-oriented: Every chat program, email,…
If all you want is knowledge about when a piece of data was created and/or modified, then tags do that just fine.
Re: Designing better file organization around tags, not hierarchies (2017)
#104Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…
[deleted]
Re: Designing better file organization around tags, not hierarchies (2017)
#105I wouldn't really call a hierarchical FS a DAG (directed acyclic graph) because of the flaws with links you've already called out. It's not a true DAG.
Are there graph-structure filesystems?
Tagging has always seemed confusing to me because it seems like a degenerate case of a graph if your tags can't contain other tags. A graph that is only two levels deep doesn't have the flexibility of a real DAG. I'm having trouble visualizing the true correspondence between tagging and a graph structure, but I think they're pretty much the same thing if you can tag tags. Does that sound right?
Finding easy fast ways to navigate graphs in various UXs (shell, file explorers, etc) is an interesting challenge. Deletion is tricky.
Re: Designing better file organization around tags, not hierarchies (2017)
#106Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…
I think a big problem with metadata-aware “file systems” is that the metadata is lost once the file is exported out of the system. This is a problem with ID3 tags for instance.
Another problem is where you make the compromise in the no-mans land between fully fledged data structure and file system. As soon as you start adding meaningful metadata to the file system, it quickly becomes apparent that you want the files themselves to be structured data and not just opaque sequences of bytes. At that point you’re redesigning the OS since that mode of usage requires user and application buy-in. It’s just a tough design problem to make any universally applicable progress in this space and it seems like any sort of non-HFS system is destined for application specific use cases.
Re: Designing better file organization around tags, not hierarchies (2017)
#107Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…
Hello, I am trying to make an application for my wife to manage embroideries. I encounter almost all your issues. My wife has thousands of embroideries downloaded from internet. There are many duplicates (filenames not unique because of internationalisation and special characters). She needs to add tags to help search. She also needs groups of tags (tiger belongs to animals, ...). She also has metadata (origin of the file, which license applies to which file). Sometimes she modifies an embroidery. She needs to ensure that the original file is not modified and to keep a link between the two files. Sometimes there are groups of embroideries (for example letters) or there is documentation attached to an embroidery. It is a mess and I think that your work would help a lot to handle this kind of use cases. The current paradigm of directory tree is outdated and something smarter could be done.
Re: Designing better file organization around tags, not hierarchies (2017)
#108Hello everyone, thank you for all the comments. Seeing this on the HN front page caught me by surprise. In the past year I shared this article publicly (Reddit) and privately (with tech-savvy acquaintances) for comment, and the general sentiment I received was that these ideas were not ready to be read by a mass audience. The article is way too long and pulls in many disparate ideas; it explains both why traditional…
I would recommend looking at some of the ideas that ReiserFS was trying to do -- and some of the ideas about metadata and expanding some of the concepts of filesystems were present in their ideas as well. In particular one idea they had was allowing you to do things like SQL searches in your filesystem, using filesystem plugins (effectively the idea is to allow for database structures to be stored using filesystem plugins so that searches and operations become just VFS operations).
Obviously we know what happened to Hans Reiser, but I've always felt that some of the clever ideas in reiser4 were not fully explored because of what happened.
Re: Designing better file organization around tags, not hierarchies (2017)
#109Earlier quoted context omitted.
Tags are fun when you have a few thousand items to test your MVP with. It gets much less fun when you have millions of items with thousands of tags, all on a flat hierarchy. On the other hand, when you're stuck with a flat hierarchy anyway (e.g. thousands of pictures, all named DCIMxxxx.jpg), tags can be more useful. But only if they're automatic. I want the best of both worlds. I want to organize my stuff into folde…
I understand your concerns, and they are indeed valid. First off, I doubt that managing millions of files in a traditional hierarchical file system is fun either. You'd likely run into problems with making unique names, sharding folders, and categorizing files that logically belong in multiple places. I also have some worries that existing file systems (say NTFS or XFS) will behave or perform well with millions of fi…
> I also have some worries that existing file systems (say NTFS or XFS) will behave or perform well with millions of files.
I believe the mantra for XFS is "if you have large or lots, use XFS". XFS has a lot of optimisations for metadata operations which should mean it's better than most filesystems for lots-of-files and large-files cases (Dave Chinner has given several talks about the performance characteristics of XFS with "large or lots" cases).
Re: Designing better file organization around tags, not hierarchies (2017)
#110An approach I find interesting is the [Perkeep]( https://perkeep.org/ ) or Google Drive model for post hierarchy. Storing all files as objects and then indexing.. An interesting indexer for images would be one that groups objects by faces recognized or exif data(camera model, GPS location, lens, date, etc) Google Drive does this. Perkeep can deal with tags, span devices, deal with permissions. Check out HN user @brad…
My main issue with Perkeep is that a lot of the automated tags are very limited, and adding more is not really easy-to-do. Though the last time I used it, it was known as Camlistore. So maybe things have changed (there were some pretty bad UI issues back then as well).
I do like the idea that nothing is deleted and everything is stored using "permanodes" and signatures of objects defining mutations of a "permanode". The downside is that everything is so incredibly dependent on the indexer, and my experience is that if the indexer has a bug you are in a lot of trouble. Also sometimes you don't want to keep everything you made 10 years ago around -- especially if it's burning storage space that costs you money.