Why I link to Wayback Machine instead of original web content • Hacker News Board

Re: Why I link to Wayback Machine instead of original web content

#131

Post by peq » Tue, Sep 08, 2020, 2:03 PM UTC

moonchild wrote: post #29

Lex-2008 wrote: post #13

Earlier quoted context omitted.

WayBackMachine alternative, archive.is, has an option to download zip archive of HTML with images and CSS (but no JS) - this way you can preserve and host a copy of original webpage on your own website

Or just wget -rk... Mirroring a website isn't so hard that you need a service to do it for you. Your browser even has such a function; try ctrl-s.

I would be careful in mirroring a site. It's very likely to violate copyright or similar laws, depending on where you are. I think archive.org is considered fair use, but if you put it on a personal or even business page it might be different. For example Google News in EU is very limited in what content they may steal from other web pages.

Re: Why I link to Wayback Machine instead of original web content

#132

Post by sebastianconcpt » Tue, Sep 08, 2020, 2:06 PM UTC

Clever way to make the reference immutable.

Some blockchain will end up taking care of this.

Re: Why I link to Wayback Machine instead of original web content

#133

Post by runxel » Tue, Sep 08, 2020, 2:12 PM UTC

While I certainly wouldn't do this with every page and also not every time, I got so anxious of link rot lately I save out of reflex any good content I come across to the Waybackmachine.

The use of the bookmarklet makes this really convenient.

Re: Why I link to Wayback Machine instead of original web content

#134

Post by spurgu » Tue, Sep 08, 2020, 2:12 PM UTC

I think a good solution might be to host the archive version yourself (archive.org is slow, and always using it centralizes everything there).

Let's say you write an article on your site, https://yoursite.com/my-article, and from it you want to link to an article https://example.com/some-article

You then create a mirror of https://example.com/some-article to be served from your site at https://yoursite.com/mirror/2019-09-08/some-article (put /mirror/ in robots.txt and set to noindex (or maybe even better to put a rel="canonical" towards the original article?)) and on the top of this mirrored page you add a header bar thingy containing a link to the original article, as well as one to archive.org if you so want.

tl;dr instead of linking to https://example.com/some-article you link to https://yoursite.com/mirror/2019-09-08/some-article (which has links to the original)

Re: Why I link to Wayback Machine instead of original web content

#135

Post by ta8908695 » Tue, Sep 08, 2020, 2:12 PM UTC

dredmorbius wrote: post #57

phendrenad2 wrote: post #34

Earlier quoted context omitted.

I wish there were a way to get a low-rez copy of their entire archive. So, only text, no images, binaries, PDFs (other than PDFs converted to text which they seem to do). As it stands the archive is so huge, the barrier to mirroring is high.

Agreed. When scoping out the size of Google+, one of ArchiveTeam's recent projects, it emerged that the typical size of a post was roughly 120 bytes, but total page weight a minimum of 1 MB, for a 1% payload to throw-weight ratio. This seems typical of much the modern Web. And that excludes external assets: images, JS, CSS, etc. If just the source text and sufficient metadata were preserved, all of G+ would be startl…

The external assets for a page could be archived separately though, right? I would think that the static G+ assets: JS, CSS, images, etc. could be archived once, and then all the remaining data would be much closer the 120B of real content. Is there a technical reason that's not the case?

Re: Why I link to Wayback Machine instead of original web content

#136

Post by liability » Tue, Sep 08, 2020, 2:15 PM UTC

bad_user wrote: post #90

jazzyjackson wrote: post #77

Earlier quoted context omitted.

Brave browser has this built in, if you end up at a dead link the address bar offers to take you to wayback machine. http://blog.archive.org/2020/02/25/brave-browser-and-the-way...

This was first implemented in Firefox, as an experiment, and is now an extension: https://addons.mozilla.org/ro/firefox/addon/wayback-machine_...

I used this extension for a while but had to stop due to frequent false positives. YMMV

Re: Why I link to Wayback Machine instead of original web content

#137

Post by lizardmancan » Tue, Sep 08, 2020, 2:20 PM UTC

also something to take home from this is that we all think to have an idea what the www is or amounts to while in reality it is changing all the time at a much more dramatic rate than we can see or indeed imagine. Depending on current events very large numbers of new sites are created and new top (of index) content is written and an even larger amount vanishes. When the new topics mature and its angles are reasonably fleshed out the incineration wave kicks in again and POOF we have a whole new www. After all, since most content is rarely linked to the problem is much larger. Naive people think the value of content is also static. They can of course advertise that opinion but for the rest of us to just accept it as ghospel??? We should be outraged so that "they" can delete it. Then we can truly feel the totalitarianism of it.

Re: Why I link to Wayback Machine instead of original web content

#138

Post by ffpip » Tue, Sep 08, 2020, 2:27 PM UTC

bad_user wrote: post #93

ffpip wrote: post #10

You can create a bookmark in Firefox to save a link quickly. Bookmark Location- https://web.archive.org/save/%s Keyword - save So searching 'save https://news.ycombinator.com/item?id=24406193 ' archives this post. You can use any Keyword instead of 'save'. You can also search with https://web.archive.org/*/%s

Does that `save` keyword work? The problem is %s gets escaped, so Firefox generates this URL, which seems to be invalid: https://web.archive.org/save/https%3A%2F%2Fnews.ycombinator....

web.archive.org automatically converts the https%3A%2F things to https:// for me. I noticed it many times.

If you are still facing problems, go to https://web.archive.org . In the bottom right 'Save page now' field, right click and select 'add keyword for search'. Choose your desired keyword.

Re: Why I link to Wayback Machine instead of original web content

#139

Post by dredmorbius » Tue, Sep 08, 2020, 2:29 PM UTC

ta8908695 wrote: post #135

dredmorbius wrote: post #57

Earlier quoted context omitted.

Agreed. When scoping out the size of Google+, one of ArchiveTeam's recent projects, it emerged that the typical size of a post was roughly 120 bytes, but total page weight a minimum of 1 MB, for a 1% payload to throw-weight ratio. This seems typical of much the modern Web. And that excludes external assets: images, JS, CSS, etc. If just the source text and sufficient metadata were preserved, all of G+ would be startl…

The external assets for a page could be archived separately though, right? I would think that the static G+ assets: JS, CSS, images, etc. could be archived once, and then all the remaining data would be much closer the 120B of real content. Is there a technical reason that's not the case?

In theory.

In practice, this would likely involve recreating at least some of the presentation side of numerous changing (some constantly) Web apps. Which is a substantial programming overhead.

WARC is dumb as rocks, from a redundancy standpoint, but also atomically complete, independent (all WARCs are entirely self-contained), and reliable. When dealing with billions of individual websites, these are useful attributes.

It's a matter of trade-offs.

Re: Why I link to Wayback Machine instead of original web content

#140

Post by cpcallen » Tue, Sep 08, 2020, 2:30 PM UTC

This seems like a risky strategy, what with the pending lawsuit against archive.org over their National Emergency Library: I am fully expecting that web.archive.org will go away permanently within a few years.