Live data from Hacker News

Link rot and content drift are endemic to the web

theatlantic.com

171–180 of 220 posts

Re: Link rot and content drift are endemic to the web

#171

Earlier quoted context omitted.

you're worried about....the ads? Having ads around doesn't make the content any less valuable. We're talking about the content itself still being available, who cares if there's some ads keeping the system up if all those live concerts and how-to vids are preserved forever

Who is going to watch live concert footage with ads jammed into it? For those of us who grew up in the age of TV, advertising was clearly a slippery slope where they constantly increased the ad content until it was beyond unbearable (they even deleted parts of shows to make room for more ads!) YouTube will likely do the same when they decide to force all of us to watch unblockable and unskipable ads to access content…

Commercials are also important media that must be archived. I watch some old TV show, old TVCMs that reflects old days are also interesting to see.

Re: Link rot and content drift are endemic to the web

#172

We need to accept that link rot and content drift are part of the web. And realize that the best place to preserve history is the Internet Archive' Wayback Machine. Kind of the same way newspapers were never responsible for maintaining their archives, but librarians did on microfiche (remember that?). But I'd take it farther. First, the Internet Archive ought to have an official partnership with the Library of Congre…

Newspapers kept extensive archives, they called them 'the morgue', and depending on the newspapers and era, they either had microfilm/fiche or actual physical clippings. When I was studying journalism in college I would get access to all the newspapers morgues. Really anyone could call up the newspaper and ask for something from a past issue, although if they needed to actually do much searching there were other hurdles. They weren't public libraries, but they served their communities.

ETA https://en.wikipedia.org/wiki/Morgue_file

Re: Link rot and content drift are endemic to the web

#173

We need to accept that link rot and content drift are part of the web. And realize that the best place to preserve history is the Internet Archive' Wayback Machine. Kind of the same way newspapers were never responsible for maintaining their archives, but librarians did on microfiche (remember that?). But I'd take it farther. First, the Internet Archive ought to have an official partnership with the Library of Congre…

Newspapers kept extensive archives, they called them 'the morgue', and depending on the newspapers and era, they either had microfilm/fiche or actual physical clippings. When I was studying journalism in college I would get access to all the newspapers morgues. Really anyone could call up the newspaper and ask for something from a past issue, although if they needed to actually do much searching there were other hurd…

And of the eight external links on that page, four are now broken.

Re: Link rot and content drift are endemic to the web

#174
post #69

Earlier quoted context omitted.

On the bright side, using a tool like Internet archive it should be easy to filter out which articles were removed and/or edited by the BBC, in a way highlighting the most historically important articles.

I mean yes, if you are extremely motivated. But the wayback machine is pretty klunky and slow, honestly. And there's no good "diff" view that summarizes the changes to a URL over time AFAIK.

We've got a pretty solid diff view now, notes at https://blog.archive.org/2019/10/18/the-wayback-machine-figh... and an example at https://web.archive.org/web/diff/20170118202526/201701200403...

Re slowness, we're able to do a lot with a little, but there's always room for improvement. If you're interested in some of the specific infrastructural challenges, I did a presentation in February:

https://archive.org/details/jonah-edwards-presentation

and my colleague did a fantastic presentation detailing some of the internal workings of the Wayback Machine just last week:

https://archive.org/details/bridget-bell-presentation

Re: Link rot and content drift are endemic to the web

#175

Earlier quoted context omitted.

I mean yes, if you are extremely motivated. But the wayback machine is pretty klunky and slow, honestly. And there's no good "diff" view that summarizes the changes to a URL over time AFAIK.

This is true, but they're also taking on an absolutely monumental task on a shoestring budget. I continue to be amazed at what they are able to accomplish. There aren't many heroes on the internet, but the Internet Archive team qualifies.

Thank you! We're doing what we can with what have where we are, or maybe just what we must because we can, depending on your preferred aphorism :)

Re: Link rot and content drift are endemic to the web

#176

Earlier quoted context omitted.

I mean yes, if you are extremely motivated. But the wayback machine is pretty klunky and slow, honestly. And there's no good "diff" view that summarizes the changes to a URL over time AFAIK.

We've got a pretty solid diff view now, notes at https://blog.archive.org/2019/10/18/the-wayback-machine-figh... and an example at https://web.archive.org/web/diff/20170118202526/201701200403... Re slowness, we're able to do a lot with a little, but there's always room for improvement. If you're interested in some of the specific infrastructural challenges, I did a presentation in February: https://archive.org/detail…

[deleted]

Re: Link rot and content drift are endemic to the web

#177

Earlier quoted context omitted.

I mean yes, if you are extremely motivated. But the wayback machine is pretty klunky and slow, honestly. And there's no good "diff" view that summarizes the changes to a URL over time AFAIK.

We've got a pretty solid diff view now, notes at https://blog.archive.org/2019/10/18/the-wayback-machine-figh... and an example at https://web.archive.org/web/diff/20170118202526/201701200403... Re slowness, we're able to do a lot with a little, but there's always room for improvement. If you're interested in some of the specific infrastructural challenges, I did a presentation in February: https://archive.org/detail…

Do you have statistics on what fraction of the pages on major outlets you have archived, and how often they change?

Re: Link rot and content drift are endemic to the web

#178

Earlier quoted context omitted.

Wayback Machine censors many websites(like 4chan) from being 'saved'. Wayback Machine also removes previously archived videos/websites in certain cases. They are not neutral.

Don’t spread misinformation. The Wayback Machine is not censoring 4chan. 4chan is ‘censoring’ the Wayback Machine. https://www.4chan.org/robots.txt Also, I let a domain of mine expire and the new domain owner (which just plastered ads) had a robots.txt that retroactively removed my “previously archived website” from the Wayback Machine.

4chan asks the WM to censor it, but the WM actually does it.

Re: Link rot and content drift are endemic to the web

#179

Earlier quoted context omitted.

How many how to videos, memes, concerts, etc, are really that important? In fifty years how many people will care? How many people should care because it would mean ignoring the huge volume of newer stuff? A hundred years? Two hundred? I haven't even read or seen many of the existing cultural artifacts we have from past decades and centuries, what would I do if orders of magnitudes more of them had been preserved? In…

These subjective questions are pointless to try and answer. The idea is that a future person could freely deep dive through a rich well indexed history of media about whatever specifically interests them I wish people would stop trying to assess the value of a given piece of media and just tag and archive the stuff. For instance, high quality footage of live music from 100 years ago would be very interesting to some.

The tagging is even harder than the storage. There might well be high quality footage of live music from not much less than 100 years ago sitting on film reels in a shed somewhere - my university's library had a whole basement of pre-1850 books that they just hadn't had time to catalogue yet.

Re: Link rot and content drift are endemic to the web

#180
Part of it is technical. The most modern stack usually completely break deep linking:

- even the best spa out there often barely shim the normal browser behavior. Yesterday my back button broke once more, in 2021. Infinite scroll don't let you pinpoint your position. User don't expect being able to copy / paste a link to take them to the content anymore.

- the ecosystem of url handling is fractured. This month I worked on a Django + React app, and my clients asked that it should be able to handle being hosted behind an arbitrary URL prefix if provided in the conf. Here are the things I had to tweak:

    * adding the prefix to the proxy pass apache conf (yes, they are still using it);
    * adding the prefix the react router conf for which most tutorials were outdated; 
    * adding the prefix in the js bundler conf as the base, and for the dev proxy;
    * adding the prefix in the  element in the main template;
    * making all urls and ajax calls relative to the ;
    * making all the react router Link and history.push _absolute_ (took me a while to figure this out);
    * serving the index.html as a template file from django, not nginx, to inject all that stuff according to the env var.
    * hacking the build script to replace static files URLS with template place holders because the js bundler didn't have a hook for that (thanks sed);
 
And that's on top of the regular work of making urls in SPA works, which implies sync your backend and frontend URLS for pages and API. Who is going to do all that works? In fact, how many devs have the knowledge to do that? Pre-SPA, there would have been well documented 2 steps to do the same thing. The junior in the team could figure it out.

- we had a ton of manure on top of our urls. AMP. Url shorteners. Tracking ID and redirections. Content wall. Captcha. Often several of them at the same time. If one of them break in the chain, goodbye URL.

- low code mean low skill devs, that never heard the mantra "cool url don't change". They don't even know they should care.

- some browsers just hide the URL. The users don't know what an url is anyway.

- apps don't care about deep linking. They could handle url fine, mind you. We have the tech for it. But it's not even on the radar of most devs. You don't address the content, you consume whatever pops up, so why bother ?

Plus, google is so good at finding the content you want out of the barely readable drunken mess of letters you feed it that most people don't type url anymore. People don't care about URL just like people don't care about bees dying, because it's too abstract to worry about.

Post reply on HN