Internet Data Is Rotting

It should still be at the discretion of the reader of what is correct and valid. In schools we are given the opportunity to learn how to critically think, vet sources, and how to seek out expert authorities on a matter (whether people practice these skills after their essay writing days are done is another can of worms). There is a lot of out of date material in libraries, for instance, but there is value in archives. Plus, who is going to copy a niche forum thread and repost it elsewhere? Most people just paste a link when they share information on the web, not clumsily copying and pasting full content. I bet whoever originally wrote that guide over a decade ago has no idea that it's rotted and useless. That information is gone unless someone knowledgeable spends their time crafting another guide, repeating the work that another already did a decade ago.

There is value in archiving things beyond just relying on the small % of things that get actively reposted elsewhere. A good local caching solution of everything you've viewed on the web would be extremely useful for your personal reference, especially when sites frequently close and links die.

If you really wanted to fantasize you could seed your cached site data using bittorrent to maintain a common web archive with everyone on the internet, free from a reliance of one lumbering archival corporate entity that might disappear on the whim of a VC or shareholder, free from commercial profiteering practices, free from the ad men and anyone else who's job it is to pollute and dilute technology for profit, and firmly in the hands of the users of the internet.

James Burke, better known for his Connections series, also produced on called "The Day the Universe Changed". In it he explores how the changing understanding, and sometimes simply views of the world, humans' place in it, and what is right and wrong, change the Universe (or at least our perception of it) itself.

One segment from the final episode featured on HN a few weeks ago. It references the burning of witches, in Scotland, from earlier in the same episode. Burke's point was that a set of beliefs which had once carried legal force no longer does. That route can run in reverse, and it's quite possible for beliefs, or actions, previously legally sanctioned (or at least not criminalised) to become otherwise. Prohibitions against ex post facto laws limit this in many jurisdictions, but changes in social mores are less forgiving. I've lived long enough to see some remarkable changes along several such lines, and have witnessed social and political upheavals elsewhere that are even more pronounced.

There's the question of whether or not it's the fact of a durable record, or the abuse of power through that record, that is the real root of the problem. As Aesop noted, any excuse will do for a tyrant. But the persistent record does seem to pose certain problems. Particularly against a shifting set of judgements.

Can you think of an even remotely feasible solution? As long as we have the technology, I don't see this stopping.

The recent shutdown of Google+ was another case of this.

As one of the people helping coordinate information among those still using the platform and hoping to migrate off of it, discovering the Archive Team's GoogleMinus project this past January was a huge boost. That ended up being the largest archival project undertaken to date, 1.6 PB, and succeeded in capturing 98% of all G+ profiles, now stored at the Internet Archive.[1]

While it had long been obvious that the project was ill-stared, the shutdown announcement came as a surprise, and Google's tools, communications, and support for both individuals, and far more importantly, groups, looking to continue their existence off the platform, was abysmal.

I don't fault Google for killing the service -- I was suprised it survived as long as it had. I do fault Google for how they did so. And that episode was hardly the worst in history.

One of the lesser-known parts of G+ were its Communities. In the process of the shutdown we came to realise that there were over 8 million of these, about 50,000 with 1,000 or more members, of all descriptions. Many frivilous or worse, but many also not. And all stuck in a very hard spot by Google's actions.[2]

Even preservation of individual data does very little for groups, and is one of the issues we're considering in the post mortem of the G+ mass migration, intended to be of use to others.[3]

________________________________

Notes:

1. For those preferring not to have their content archived, the IA WBM respect DMCA requests, and as Google+ posts are all listed under the user's account, requesting removal is exceedingly straightforward.

2. Characteristics of number and size are collected here, compiled by me, based in part on on data provided by Friends+Me: https://social.antefriguserat.de/index.php/Migrating_Google%...

3. Discussion at Reddit and elsewhere. Compilation at the PlexodusWiki. https://old.reddit.com/r/plexodus/comments/boa97x/g_migratio... https://social.antefriguserat.de/index.php/G%2B_Migration_Po...

Back in the mists of time, I used to use wwwoffle proxy. It was great for low-latency links, but also had the benefit of keeping an offline archive of whatever you'd browsed.

Project's still there, although not sure how well it does with the modern web.

http://www.gedanken.org.uk/software/wwwoffle/

There are a bunch of more modern variations too:

https://archivebox.io/ - "Your own personal internet archive"

https://getpolarized.io/ (as seen on HN previously)

https://github.com/kanishka-linux/reminiscence

https://github.com/fake-name/ReadableWebProxy