Small Archives
helloruby.substack.com
Small Archives
1–10 of 12 posts
Re: Small Archives
#2I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture.
What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
Re: Small Archives
#3I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
Have you looked at Camlistore?
Re: Small Archives
#4I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
I’m reasonable satisfied with Shiori and its text search functionality
Re: Small Archives
#5I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
Have you looked at Camlistore?
Now known as Perkeep (previously Camlistore)
Re: Small Archives
#6Earlier quoted context omitted.
Have you looked at Camlistore?
Now known as Perkeep (previously Camlistore) https://en.m.wikipedia.org/wiki/Perkeep
Perkeep is very interesting on a fundamental level, as a concept and as a building block, but sadly hasn't received enough love to fully fulfill its vision.
I wish there were more people organizing around Perkeep and breathing life into it.
Re: Small Archives
#7I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
I too am looking for a tool that can decently mirror sites. The next best thing is something like Instapaper or Obsidian Clipper which export copies of a site (into markdown, in the latter's case).
Re: Small Archives
#8I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
While I haven't personally tried it myself, ArchiveBox seems to be tailored specifically for archiving[1]. Mb you should give it a try?
[1]https://archivebox.io/
Re: Small Archives
#9I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
Modern hard drives are now above 20TB in capacity. If your average file size is 100KB, then they will hold 200 million+ files.
File systems with their directory or folder hierarchy are poor organizers of files. Searching for files is a huge hassle.
File systems, which were designed over 50 years ago when storage was measured in MB, need to be replaced by object stores that are much better at organizing and finding data.
Re: Small Archives
#10I've seen so many things disappear off the web that I knee-jerk yt-dlp/wget/DeDRM anything I am interested in or is relevant to my personal culture. What stinks is I haven't found anything satisfactory in terms of archival software that runs locally. For now, I shove things into folders in external drives, but directory trees are very poor when it comes to categorization and retrieval.
Modern hard drives are now above 20TB in capacity. If your average file size is 100KB, then they will hold 200 million+ files. File systems with their directory or folder hierarchy are poor organizers of files. Searching for files is a huge hassle. File systems, which were designed over 50 years ago when storage was measured in MB, need to be replaced by object stores that are much better at organizing and finding da…
In what way are object stores better than hierarchical filesystems? With links you can create arbitrary indices over the same data with different (or the same) names.