Live data from Hacker News

This Page is Designed to Last

jeffhuang.com

251–260 of 458 posts

Re: This Page is Designed to Last

#252

Earlier quoted context omitted.

This problem has been solved a long time ago if you use Pinboard. https://pinboard.in/ Just pay the yearly subscription so pinboard can cache your bookmarks.

Does pinboard caching work with sites that are behind a login or paywall?

It does not. See FAQ here: https://pinboard.in/upgrade/

Re: This Page is Designed to Last

#253
post #13

There's no reason why a web browser bookmark action doesn't automatically create a WARC (web archive) format. Heck, with the cost of storage so low, recording every webpage you ever visit in searchable format is also very realistic. Imagine having the last 30 years of web browsing history saved on your local machine. This would especially be useful when in research mode and deep diving a topic. [1] https://github.com…

How is viewing the WARC after? Is it the same quality as archive.is/ or archive.org/ ?

It's a pain, like most of the WARC ecosystem. It's been several months since I dug in, so maybe it's had some spitshine the last little bit, but I usually end up using combinations of wpull, grab-site, and a smattering of other utilities to reliably capture a page/set of pages, and have had to make some quick hacks as well as manually merging in some PRs to get things to work with Python3. Once I have the WARC, I typically end up using warcat to extract the contents into a local directory and explore that way.

WARC as a format seems promising, but at least last I checked, open-source tooling to make it a pleasant and/or transparent experience is not really there, and worse, at least as of several months ago, doesn't really seem actively worked on. Definitely an area you'd expect to be further along.

Re: This Page is Designed to Last

#254

We need a common distributed storage protocol - kind of Bit Torrent for web pages. Dead simple in use and built into (every) browser transparently. Anytime a public page is visited/bookmarked then it is preserved in a swarm of caches until the last seeder dies.

IPFS aims to be this, but it's still not working well enough.

Re: This Page is Designed to Last

#255
post #209

Earlier quoted context omitted.

This problem has been solved a long time ago if you use Pinboard. https://pinboard.in/ Just pay the yearly subscription so pinboard can cache your bookmarks.

I do not see how using a web thing is a solution to web things going away.

Pinboard happens to be a web service run along the same principles as the article we're discussing.

The bus factor is high, but I suspect that Maciej has a plan that'll let us download our archive even if he does get grabbed by the mainland Chinese government let alone a forecasted going out of business action.

Re: This Page is Designed to Last

#256
post #142

Earlier quoted context omitted.

The Internet Archive is a fantastic resource. And right now, they happen to match every donation two to one (so $5 becomes $5 + 2 * $5 = $15 dollars!)

How do they match a donation to themselves?

They currently have a deal with a donor who will donate $2 for every $1 the Archive gets in that time period.

Re: This Page is Designed to Last

#257

Earlier quoted context omitted.

I store bookmarks (i.e. URLs) in a simple .txt file. My text editor lets me click on them to bring them up in a browser. > Only downside is I'm dependent on Evernote No special software nor database required.

What benefit does that provide compared to regular browser bookmarks? It doesn't seem to address either of the issues I mentioned.

1. it's independent of the browser

2. it works with any browser

3. I can move it to any machine and use it

4. It is not transmitted to the browser vendor

5. Being a text file, it is under my control

6. I can back it up

7. I don't need some database manager to access it

8. I can add notes and anything else to the file

9. It's stupid simple

Re: This Page is Designed to Last

#258

I completely agree with this article, however ... > (4) End all forms of hotlinking Any advice on what to do if you want to embed videos? Bandwidth can cost a lot (maybe I'm wrong?) and AFAIK Cloudflare doesn't cache videos. I would stick with Vimeo, but they can delete your videos if over threshold after you stop paying. YouTube at this point is sadly a better bet if you want to maintain a low cost website for decad…

If bandwidth (and not storage) is the only issue, you could upload it on youtube but still add a direct link to your self-hosted file.

Re: This Page is Designed to Last

#259

There's no reason why a web browser bookmark action doesn't automatically create a WARC (web archive) format. Heck, with the cost of storage so low, recording every webpage you ever visit in searchable format is also very realistic. Imagine having the last 30 years of web browsing history saved on your local machine. This would especially be useful when in research mode and deep diving a topic. [1] https://github.com…

This was what made me convert from bookmarking to clipping pages into Evernote around 6-7 years ago. I realized I had this huge archive of reference bookmarks that were almost useless because 1) I could rarely find what I was looking for, if I even remembered I'd bookmarked something in the first place, and 2) if I did, it was likely gone anyway. With Evernote I can full text search anything I've clipped in the past…

I'm glad you mention Evernote. I also use it for this, and also for many other purposes.

It is true that it is propietary software but it is worth mentioning that all the content can be exported as an .enex file, which is xml.

So, the data can be easily exported.

Re: This Page is Designed to Last

#260
post #124

Let's be honest with ourselves. The best way to make your content last for a long time is to host it on a platform that is free and very successful. For example, whatever photos I posted on Facebook 12 years ago? Still alive and kicking. The articles I've published on wordpress.com 7 years ago? Still in mint condition, with 0 maintenance required. In comparison, the websites that I've built and hosted or deployed mys…

>to host it on a platform that is free and very successful

Yes, from a technical pow, but what about deplatforming? I think it is a bigger risk to lose data than any framework/technology deprecation. I would definitely not rely on any platform keeping my data.

Post reply on HN