Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sum of all knowledge and the sorry state of the web

christianheilmann.com

Post reply on HN
Print view View original

131–140 of 187 posts

Re: The sum of all knowledge and the sorry state of the web

#131

Post by softcactus »

mellavora wrote: post #32
Eleison23 wrote: post #14

Earlier quoted context omitted.

>Google started as a DARPA project, and was a great engine while it stayed true to that ethos. It was the need to commercialize it, thus setting perverse business incentives, which has destroyed it. Google started as a what now? This is an interesting thing to say during a discussion of access to information. I'd be happy to read your explanation of Google's "DARPA roots", and your citations to sources explaining how…

It is indeed a fascinating read. Here you go! https://qz.com/1145669/googles-true-origin-partly-lies-in-ci... >A second grant—the DARPA-NSF grant most closely associated with Google’s origin—was part of a coordinated effort to build a massive digital library using the internet as its backbone. Both grants funded research by two graduate students who were making rapid advances in web-page ranking, as well as tracking…

I always wondered how their initial python web scraper was fast enough to index the entire internet on that old hardware, even given the small size of the internet at the time. I guess the answer is that they had a local backup at Stanford. Thanks for sharing!

Re: The sum of all knowledge and the sorry state of the web

#132

Post by crazygringo »

> Except, it isn’t any longer. The web we have these days is in a sorry state.

The web is in a better state than it's ever been. Just between Wikipedia, Library Genesis and Sci-Hub alone, let alone things like Google Search, Google Scholar, or all the content put online from various museums/libraries/etc.

> On top of that drowning in memes, reposts and funny things you already read in newsgroups in 1998.

None of that takes away from all the good parts.

I honestly don't get any of these posts about how the web is so bad or so much worse than it used to be. The absolute amount of quality content has grown, seemingly exponentially. Even if the relative amount of quality has decreased... who even cares?! You don't have to visit the parts of the web you don't like.

And if you're complaining about reposts...? I have zero sympathy for you. Lots of people are seeing things for the first time. And it's nice to be reminded of things too.

Re: The sum of all knowledge and the sorry state of the web

#133

Post by fatneckbeardz »

i think to change this, you have to start with a basic, fundamental respect for the liberal arts, the humanities, and for human beings in general. and that means you have to pay people in money for the emotional and mental labor of organizing information. and stop trying to replace them with algorithms to maximize clicks and watchtime.

Re: The sum of all knowledge and the sorry state of the web

#134

Post by Melatonic »

nicbou wrote: post #5

It has never been easier to share knowledge, and thus there has never been a greater time to be curious. Encyclopaedia Britannica is still there, and it's great . But there's also an army of content creators are there to teach you just about anything. If one source doesn't do it, you have more to choose from. Yesterday, I read an article about North Africa that mentioned in passing Qaddafi's underground river. A few…

Lets face it - early libraries probably sucked too. We will improve

Re: The sum of all knowledge and the sorry state of the web

#135

Post by vsareto »

sandruso wrote: post #12
mudrockbestgirl wrote: post #6

IMO the worst side effect of current web of knowledge is what I'd call the illusion of knowledge. When it was more difficult to access and publish information, that imposed a much higher bar on what was being consumed. These days, people watch a 10-minute YouTube video or read a reddit comment or twitter thread and believe (perhaps unconsciously) that makes them knowledgeable in said topic. They will then, in an abso…

Its good to remind myself this . Not once I thought I know something after watching a summary video. When I tried to explain the topic to somebody else I struggled. If you can’t explain you know nothing. Simple as that.

>If you can’t explain you know nothing. Simple as that.

This is still a pretty good bar, but if I just repeat what the video said, it seems like I'm explaining it, and then I can give the impression I do know something. I shouldn't get credibility for just repeating a video.

The US has devolved into somewhat of a reputation-driven expertise market, and there are plenty of ways to gain reputation without the expertise. There's still plenty of real experts, but I don't blame them if they want to focus on their work instead of fighting the endless tide of easily produced misinformation.

Re: The sum of all knowledge and the sorry state of the web

#136

Post by antonymy »

I definitely worry about the future of scholarship as our print media becomes more and more fragmented and hidden. It's not necessarily about ensuring essential knowledge is carried forward, but rather a "sense of the past". In researching history, the further back you go the more fragmented and unreliable your sources usually become. So it becomes harder and harder to figure out the broad sweep of these past cultures, what life was like, the things they believed, and so on. And I think we're doing that to our present historical moment by deleting the recent past and creating a continuous present.

We have the Internet Archive, we have Wikipedia page histories, but everything else is ephemeral. If Archive.org ever goes the way of the Library of Alexandria, we'll have lost irreplaceable knowledge of the web itself, and the cultures that existed on it. It will live on only in living memory, but this is also transient, and soon nobody will live who remembers what this time period was like. Wikipedia will not reflect it, you'd have to dig through page histories to find fragments, like historians and archaeologists sifting through ancient manuscripts and ruins for clues.

A potential dark age is forming, and avoiding it right now hinges entirely on the continued efforts of two donation-funded organizations, one of which makes it increasingly harder to view the past, the other facing legal disputes that could see it shut down. I think we need to make archiving the present important, so it does not become a mysterious, inscrutable past for our descendants.

Re: The sum of all knowledge and the sorry state of the web

#137

Post by hinkley »

lordnacho wrote: post #36
nicbou wrote: post #5

It has never been easier to share knowledge, and thus there has never been a greater time to be curious. Encyclopaedia Britannica is still there, and it's great . But there's also an army of content creators are there to teach you just about anything. If one source doesn't do it, you have more to choose from. Yesterday, I read an article about North Africa that mentioned in passing Qaddafi's underground river. A few…

It's really the curation that needs to be taught to everyone. A big dose of critical thinking skills is what we all need, because in earlier times you could tell the crappy ideas by the way they were packaged: crazy guy at Hyde Park Corner, dude with a megaphone shouting out passages of the bible, crappy home-made flyers. You could work backwards from "nutter is probably wrong" to why he was wrong. Part of why this w…

Curation is the first step, but we also need more organizations who take on the job of keeping some bits of the internet from going dark. Especially for things that were originally done for the common good.

There's a local group here that basically specializes in devops for little public projects that have run their course. They even do a little bit of work trying to provide templates for new projects (eg, for a local hackathon), but I'd like to see them go farther.

Bigger, more national or international groups that come up with recipes for projects where they say, "If you build your project on this structure, then we will be more likely to run it for you," I think is a reasonable logical next step for the Internet. Given the task of running 10 projects that 'need' 4 servers each, it would be very good if I could do it all with 20 servers, not 44 (40 + orchestration machines).

We don't have a "PBS for the internet" but then PBS didn't always exist either. You need a beach head of some sort to even propose such a thing to government.

Re: The sum of all knowledge and the sorry state of the web

#138

Post by naasking »

imiric wrote: post #10
nicbou wrote: post #5

It has never been easier to share knowledge, and thus there has never been a greater time to be curious. Encyclopaedia Britannica is still there, and it's great . But there's also an army of content creators are there to teach you just about anything. If one source doesn't do it, you have more to choose from. Yesterday, I read an article about North Africa that mentioned in passing Qaddafi's underground river. A few…

> It has never been easier to share knowledge, and thus there has never been a greater time to be curious. It's also never been easier to share disinformation, and pollute the vast sea of actual knowledge that exists on the internet. Sources of information are silo'd into proprietary closed off gardens run by large corporations who only serve their shareholders. Searching for information has been corrupted by adverti…

> Sources of information are silo'd into proprietary closed off gardens run by large corporations who only serve their shareholders. Searching for information has been corrupted by advertisers and the sheer amount of misleading content, that finding reliable sources often feels like searching for a needle in a haystack.

I'm not particularly worried about disinformation or misinformation as long as good information is out there to improve the signal to noise ratio. But you've definitely hit a nail on the head here, good information has value so there are a lot of incentives to wall it off and charge for it. This is the real danger.

Re: The sum of all knowledge and the sorry state of the web

#139

Post by Wowfunhappy »

VHRanger wrote: post #90
grumbel wrote: post #50

Earlier quoted context omitted.

> No amount of YouTube videos can replace that. The irony here is that Youtube videos from ten years ago are still alive and well. As Youtube makes a much better places for publishing and archiving content than the rest of the Web. With Youtube you don't have to worry about URLs changing or domain names expiring or anything like that. You just publish your video once, get a unique video-id and don't have to worry abo…

> The irony here is that Youtube videos from ten years ago are still alive and well. They're not, though. Because Youtube re-encodes videos every couple of years, with new "better" lossy compression algorithms. And each time the videos get successively worse. Watching a 2008 Youtube video will not only look grainy because it's 360p, but it'll look actively *worse* than it did in 2008 because of all the lossy compress…

As a sibling comment noted, I'm pretty sure Youtube keeps the original video files, so generational loss should not be a huge problem.

The problem is that Youtube is bitstarving 240p and 360p streams. That makes sense when a video is available at a higher resolution, and the 240p version is for people stuck on dial up or whatever. But in cases where 240p is the highest resolution available, Youtube should provide a high quality 240p stream!

Re: The sum of all knowledge and the sorry state of the web

#140

Post by smolder »

naasking wrote: post #138
imiric wrote: post #10

Earlier quoted context omitted.

> It has never been easier to share knowledge, and thus there has never been a greater time to be curious. It's also never been easier to share disinformation, and pollute the vast sea of actual knowledge that exists on the internet. Sources of information are silo'd into proprietary closed off gardens run by large corporations who only serve their shareholders. Searching for information has been corrupted by adverti…

> Sources of information are silo'd into proprietary closed off gardens run by large corporations who only serve their shareholders. Searching for information has been corrupted by advertisers and the sheer amount of misleading content, that finding reliable sources often feels like searching for a needle in a haystack. I'm not particularly worried about disinformation or misinformation as long as good information is…

> good information has value so there are a lot of incentives to wall it off and charge for it.

There's even an incentive there to pollute free sources of good information in order to devalue them relative to proprietary knowledgebases.

Post reply on HN