The web can never be the "sum of all knowledge".
It's simply impossible. It always irks me when people refer to the web as 'all human knowledge', because it's woefully incomplete if that is the goal.
The sum of all knowledge and the sorry state of the web
171–180 of 187 posts
Re: The sum of all knowledge and the sorry state of the web
#172Earlier quoted context omitted.
I’m not sure you and the author are talking about the same thing. He mentions the fact you can’t even read the news from 10 years ago, the content has simply disappeared. No amount of YouTube videos can replace that. The problem is not “everything, everywhere” or a lack of filters but the extreme commercialization of all content available, closed networks, the short life of URLs…
You absolutely can read news from 10 even 20 even 100 years ago. One example: https://news.google.com/newspapers I think you forget or may not have grown up with the microfiche. The reality of the internet is that everyone has a voice and things will only be archived if someone gives a damn to archive them. And that's fine. Some information deserves to be transient. Hell, we've survived millennia without this level o…
That's fine for stuff that was printed in newspapers, but you (like the topmost commenter) seem to be replying to something different from what the article is talking about.
Paper is great, because it doesn't just evaporate when you look away. Although it does degrade, it's a slow process—slow enough that you can notice that it's happening and think to yourself, "Gee, I maybe ought to do something about this." It's not the same for transient digital media. There are bonafide news items and other digital content that are now no longer accessible because they were digital-first but the business incentives were so misaligned and/or their legacy has been so mismanaged that, perversely, it's easier to access to the content of a 50-year old news article than it is for others that are 5–15 years old. People can always trawl through their parents' and grandparents' belongings and come across the only known surviving copy of something and donate it to a library or sell the collection in a yard sale or eBay. (Whether they know it's the only one known to exist or not isn't a precondition.) That's not just less likely with the Web, but it's drastically less likely. No one's gleaning much from the unevicted entries in someone's browser cache.
Re: The sum of all knowledge and the sorry state of the web
#173Earlier quoted context omitted.
What is the most bizarre thing to me about the online zeitgeist is how so many people will allow comments of anonymous or pseudonymous strangers on places like Reddit or Twitter to shape their world view. Including journalists. An extreme or inaccurate view may start on social media and be normalised through repetition on social media and subsequent validation by main stream journalist. In the case of Reddit in parti…
> What is the most bizarre thing to me about the online zeitgeist is how so many people will allow comments of anonymous or pseudonymous strangers on places like Reddit or Twitter to shape their world view. Including journalists. I totally second this, and have a recent concrete example where at the beginning of this year Sweden's (probably) most serious newspaper (what I'd consider a 'journal of record' whose articl…
> I wrote and complained to the responsible editor with citations showing how and where the article was wrong, and a few very grudging emendations were made (effectively saying that even though the reports were still probably true, they couldn't be 'verified').
That's interesting, because it possibly led to a sort of well-meaning destruction. Was the content surgically edited, or did it grow a warning that some of the info originally published was incorrect, or (more perniciously) both? Consider the affect this has on researchers trying to do studies on misinformation—they certainly want to be able to access the originals themselves.
Re: The sum of all knowledge and the sorry state of the web
#174Blogs are still an underrated goldmine of knowledge, especially in tech. I find academic papers often too abstract or opaque, textbooks are good but generalized, and documentation is reference-like. Stumbling across a tech blog where someone explains some fairly specific and difficult problem they had, and an interesting solution they found, can be exactly what you needed to solve a problem. The web has its problems…
[deleted]
Re: The sum of all knowledge and the sorry state of the web
#175I definitely worry about the future of scholarship as our print media becomes more and more fragmented and hidden. It's not necessarily about ensuring essential knowledge is carried forward, but rather a "sense of the past". In researching history, the further back you go the more fragmented and unreliable your sources usually become. So it becomes harder and harder to figure out the broad sweep of these past culture…
Technologists and those in an adjacent professional class (which should account for a lot here on HN) should also do their part to help make sure that the present is easily archivable.
Re: The sum of all knowledge and the sorry state of the web
#176Earlier quoted context omitted.
From my admittedly limited understanding, the failure of the semantic web is one of mankind's biggest missed opportunities. Now the knowledge graph is just locked behind Google's neural network layers and only being used for ads.
Maybe something like a 'WikiInfo' (or another better name :) ), that contains a hierarchy of (potentially all) known pages and topics? I think the only way to tackle this problem is collaboratively and distributedly. You could add for example a 'Newspapers' topic, and then say 'The Springfield Times' and then have 'Articles by date', 'Articles by topic', etc. like a huge database (browsed hierarchically like "WHERE d…
EdgeDB is half of what you describe on the database front. You can add nested queries with filters and calculated query types. Every item in the database is given a unique UUID and there's support for complex custom types and constraints, included calculated constraints I believe.
You could have an Article type with a link to a Person as the author, and many more types of Works besides. You could find any Work by a Person traversing backlinks to find any object linked to that person. Any work where they contributed. Any social media link they posted.
Then query that by a duration of time starting from a specific date. A specific place, if it has it, a group of sites and so on.
If my understanding of it from my time playing with it is correct. I haven't experimented with too broad a dataset yet.
Re: The sum of all knowledge and the sorry state of the web
#177Earlier quoted context omitted.
> No amount of YouTube videos can replace that. The irony here is that Youtube videos from ten years ago are still alive and well. As Youtube makes a much better places for publishing and archiving content than the rest of the Web. With Youtube you don't have to worry about URLs changing or domain names expiring or anything like that. You just publish your video once, get a unique video-id and don't have to worry abo…
Unfortunately when I look through my old Favourites playlist, which at some point reached the limit of 5000 videos, I can see how many of those videos are now private, deleted, or blocked in my country. The worst part is that in many cases I can't even recover the video title, so I have no idea what has been lost. A possible solution would be to store the titles separately, but I didn't think about this while I added…
>The worst part is that in many cases I can't even recover the video title, so I have no idea what has been lost.
I use two methods to recover deleted video titles:
1) google videoId part of the URL
2) look up cleaned up (with removed playlist-related parameters, just "youtube.com/watch?v=[videoId]" form) video URL in web.archive.org - sometimes it even has an archived video itself!
Re: The sum of all knowledge and the sorry state of the web
#178Earlier quoted context omitted.
Knowledge does not imply intelligence. Nor is the opposite true.
Since Astronomy is considered one of seven Liberal Arts, knowing the very bases of orbital mechanics is not about knowing for example difference between interfaces and abstract classes. Understanding this astronomical subject speaks to a general curiosity that in my opinion correlates with intelligence. An interesting observation, this is not the first time I've talked about this little test on HN, but this time it w…
[deleted]
Re: The sum of all knowledge and the sorry state of the web
#179Earlier quoted context omitted.
> No amount of YouTube videos can replace that. The irony here is that Youtube videos from ten years ago are still alive and well. As Youtube makes a much better places for publishing and archiving content than the rest of the Web. With Youtube you don't have to worry about URLs changing or domain names expiring or anything like that. You just publish your video once, get a unique video-id and don't have to worry abo…
There's plenty of videos on youtube that have been removed. Every few years when I look through my list of liked videos, a couple more are gone forever. Granted, this is likely by the creator themselves, but that doesn't matter when the purpose is archival.
I hate creating playlists on YT for this reason.
When they delete videos, they don't even list the title of the video anymore, they remove the thumbnail and everything, so you won't even know that something you added months ago is no longer available.
Re: The sum of all knowledge and the sorry state of the web
#180Earlier quoted context omitted.
I’m not sure you and the author are talking about the same thing. He mentions the fact you can’t even read the news from 10 years ago, the content has simply disappeared. No amount of YouTube videos can replace that. The problem is not “everything, everywhere” or a lack of filters but the extreme commercialization of all content available, closed networks, the short life of URLs…
Not really. There are vast archives of newspaper articles accessible through a web interface. You just need library access. The free, open web is mostly just a spam ocean, but if you make an effort to access the services that catalog useful information, it’s still very useful. The Google web is shit, though.
> You just need library access.
Assuming you live in a place where public libraries are well-funded and can provide such amenities.