If it is worth keeping, save it in Markdown
p.migdal.pl
If it is worth keeping, save it in Markdown
1–10 of 282 posts
Re: If it is worth keeping, save it in Markdown
#2I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
Re: If it is worth keeping, save it in Markdown
#3I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line.
I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
Re: If it is worth keeping, save it in Markdown
#4I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
> If it's worth saving
Idk, I think if it's worth saving it's worth saving and the only person who can determine if it's "worth it" is me.
I agree that some people have an obsession where they save data that isn't worth it, but r/DataHoarder is a great place with a lot of information on building and maintaining large data systems for hobbyists, regardless of what you actually store.
Re: If it is worth keeping, save it in Markdown
#5I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line. I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
Until run you out of space and/or storage bill is too expensive.
Re: If it is worth keeping, save it in Markdown
#6I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
> I understand the sentiment that you can't trust content on the web to be there forever
The thing is, people say this, and I am sure for some amount of content it's true. However I eventually realised I have never had a single issue if required, in retrieving literally any piece of software or digital content after the fact from somewhere on the internet.
It's pretty much why I care so little about what happens to my steam library when Gaben kicks it. If I get the urge to replay something in twenty years that I paid $3 dollars for and its suddenly gone, i'll just go find it elsewhere.
Once I had gotten pretty much everything I wanted from running a large scale storage system (largely to learn the in's and outs of linux/general storage concepts) I pretty much just gave it up. Its a lot of money to hold onto things that at this point, I pretty much know i'll always be able to recover elsewhere. I'd rather someone else pay the electricity/drive cost for me.
Re: If it is worth keeping, save it in Markdown
#7I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
When I find a blog post/article I'm interested in, I save it to my laptop with the SingleFile extension and I take quick notes as well I write my thoughts about it in org-mode. It has a very low cognitive threshold and I can always read it back in the browser. I'm find if not all the outbound links are still working, I'd just like to read back sometimes.
Re: If it is worth keeping, save it in Markdown
#8I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.
I agree, but I also think that data hoarding is similar to regular hoarding. All of this content and information seems like it could be useful, invaluable even. It's a problem in a world where we have excess to sort through all that information and only focus on what's important right now.
Re: If it is worth keeping, save it in Markdown
#9Earlier quoted context omitted.
I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line. I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
Until run you out of space and/or storage bill is too expensive.
if you're storing text that is realistically not going to be a problem.
Re: If it is worth keeping, save it in Markdown
#10I've been doing this recently with every URL I've bookmarked over the last 15 years or so since I signed up for pinboard.in. http://spider.cloud has been really nice for crawling sites and saving the results as markdown. I plan on expanding it to transcribing youtube videos I've saved, github repos I've starred, HN posts, etc.
Ultimately I'm trying to index my "window" to the web as embedded content in a vector store. Not sure exactly what I'm going to do with it yet but I imagine it will be a component of some kind of personal agent system I can use to reference old info and help as a writing tool or as an "idea generator" of some kind. I'll likely end up not using most of it but you never know.
I've scraped about 10k markdown files which has created a ~10gb chromadb instance so far. Eventually I'll probably create separate collections based on domain, and filter down items that I care about more.