Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Locking the Web Open: A Call for a Distributed Web

brewster.kahle.org

Post reply on HN
Print view View original

51–60 of 100 posts

Re: Locking the Web Open: A Call for a Distributed Web

#51

Post by notduncansmith »

LukeB42 wrote: post #34

I've been thinking of how to decentralise the web as-is since 2011, the current development branch for this perspective on it is here: https://github.com/LukeB42/Uroko/tree/development It's basically a collaborative caching proxy. One process is a proxy that can also coordinate multiple users editing the same page and a subprocess acts as a DHT node. You can use a raft-like log of hashes of pubkey,content and the pre…

So the "log of hashes of pubkey,content and the previous hash" is conceptually similar to a blockchain, and I think reading into how that works (consensus, trust, etc) would give you some insight into the issues you're describing. You may also find the IPFS project of interest: http://ipfs.io/

Re: Locking the Web Open: A Call for a Distributed Web

#53

Post by LukeB42 »

notduncansmith wrote: post #51
LukeB42 wrote: post #34

I've been thinking of how to decentralise the web as-is since 2011, the current development branch for this perspective on it is here: https://github.com/LukeB42/Uroko/tree/development It's basically a collaborative caching proxy. One process is a proxy that can also coordinate multiple users editing the same page and a subprocess acts as a DHT node. You can use a raft-like log of hashes of pubkey,content and the pre…

So the "log of hashes of pubkey,content and the previous hash" is conceptually similar to a blockchain, and I think reading into how that works (consensus, trust, etc) would give you some insight into the issues you're describing. You may also find the IPFS project of interest: http://ipfs.io/

I've found IPFS very interesting and reccommended it to peers, but it lacks the collaborative editing aspect.

Trusting the initial public offering of a resource is still an interesting issue. IPFS is content-addressable by hash, addresses map to their content in a computable way.

The idea for the distributed hash table in Uroko is that the keys are existing URLs. Imagine thousands of peers all saying they have a new page on the domain "google.com" and you can see what makes this a fun problem to solve.

Re: Locking the Web Open: A Call for a Distributed Web

#54

Post by nekopa »

Titanous wrote: post #41
nekopa wrote: post #39

Earlier quoted context omitted.

Is that even possible? I don't know the latest size of the IA, but it must be ridiculously huge by now, (1 billion pages a week added) bandwidth cost would be massive. Maybe they could offer a mail-us-a-multi-petabyte-hdd service... Returned a few weeks later full of data :)

It's totally possible, they already have the infrastructure in place and 14PB of data available for download. Unfortunately the Wayback Machine data is not currently exposed publicly.

Why do you think that is? It seems like they are really open with most of their stuff, so why haven't they exposed the wayback with an api?

Then again, wouldn't it be pretty trivial to scrape?

(I say this as I'm working on a hellish scraping project, and the wayback machine seems like it would be a walk in the park to scrape)

Re: Locking the Web Open: A Call for a Distributed Web

#55

Post by nekopa »

genghisjahn wrote: post #47
nekopa wrote: post #37

Earlier quoted context omitted.

Not talking about the legal aspects, but, (and I really, really hate to bring up the "think about the children!" argument here) what about if I am unknowingly helping people who create and share child porn? It doesn't matter (to me) if I am on the hook for it or not, I just don't know (ethically?) how I would feel if I knew that was going on via my PC. Drugs I don't give a shit about, and I hate how the "think about…

Maybe think of it this way, if you were a construction worker and helped build an interstate, you'd be helping all sorts of criminals do all sorts of horrible things. Plus innocent people would die in car crashes on the road you helped to build. But we can agree that the utility of an interstate far outweighs these drawbacks.

I get the analogy, but it just doesn't quite... fit this problem for some reason (maybe because there are a lot of crimes I feel aren't crimes, and crashes are just random fate most of the time).

Plus I could think of numerous ways to argue against the analogy that would literally make me sound like a "think of the children!" person...

Maybe I just need to think more about my internal attitudes and justifications for certain things in a more critical, rational way.

Thank you for giving me food for thought.

Re: Locking the Web Open: A Call for a Distributed Web

#56

Post by gglitch »

Animats wrote: post #48

Kahle's approach works only for static content. It's not hard to distribute static content; BitTorrent does it just fine. The Internet Archive stores static content. Kahle thinks in terms of static content, because that's what the Internet Archive does. But it's less of the Web today. Despite that, it's good to have a way to distribute static content. Academic publishing, after all, is almost all static content. That…

Why are micropayments worse than the current web? People have differing opinions on the advertising-pays-for-content-so-don't-block-it issue, but are you referring to something technical?

Re: Locking the Web Open: A Call for a Distributed Web

#57

Post by genghisjahn »

nekopa wrote: post #55
genghisjahn wrote: post #47

Earlier quoted context omitted.

Maybe think of it this way, if you were a construction worker and helped build an interstate, you'd be helping all sorts of criminals do all sorts of horrible things. Plus innocent people would die in car crashes on the road you helped to build. But we can agree that the utility of an interstate far outweighs these drawbacks.

I get the analogy, but it just doesn't quite... fit this problem for some reason (maybe because there are a lot of crimes I feel aren't crimes, and crashes are just random fate most of the time). Plus I could think of numerous ways to argue against the analogy that would literally make me sound like a "think of the children!" person... Maybe I just need to think more about my internal attitudes and justifications for…

I'm still chewing on this myself. The previous interstate analogy was more of a, "I think I think this."

Re: Locking the Web Open: A Call for a Distributed Web

#58

Post by nekopa »

joesmo wrote: post #43
nekopa wrote: post #37

Earlier quoted context omitted.

Not talking about the legal aspects, but, (and I really, really hate to bring up the "think about the children!" argument here) what about if I am unknowingly helping people who create and share child porn? It doesn't matter (to me) if I am on the hook for it or not, I just don't know (ethically?) how I would feel if I knew that was going on via my PC. Drugs I don't give a shit about, and I hate how the "think about…

With a fully distributed model, everyone is essentially running a backbone server. If you don't feel comfortable with such an arrangement, then you'd probably have to opt out. There are plenty of people willing to put up with it, evidenced by the number of companies who have no problem operating the current Internet backbone despite knowing for a fact that their networks are used to distribute child porn and other il…

I do feel comfortable with being a part of a global backbone server, and you're right, I don't see a way around the moral issue. But I do see a light at the end of the tunnel in the idea of serving what I consume, that is an interesting take on it. But that does also bring up the issue of creating walled gardens (ie niche bubbles) that may not foster the type of Internet I want to see in the future...

For me it's a thorny issue just because of recent articles I've read about the anonymous services really allowing molesters to even set up 'dating' services. As the post above argues about utility outweighing the bad, I want the future great possibilities of an unhindered Web, but I don't want the unheard of new ways people can abuse children. (I've stated before that there are a lot of crimes I don't care about too strongly, but this area gets me, mainly because I have small kids right now)

I guess I just want to have my cake and eat it...

Re: Locking the Web Open: A Call for a Distributed Web

#59

Post by nekopa »

genghisjahn wrote: post #57
nekopa wrote: post #55

Earlier quoted context omitted.

I get the analogy, but it just doesn't quite... fit this problem for some reason (maybe because there are a lot of crimes I feel aren't crimes, and crashes are just random fate most of the time). Plus I could think of numerous ways to argue against the analogy that would literally make me sound like a "think of the children!" person... Maybe I just need to think more about my internal attitudes and justifications for…

I'm still chewing on this myself. The previous interstate analogy was more of a, "I think I think this."

Fair enough.

Re: Locking the Web Open: A Call for a Distributed Web

#60

Post by dublinben »

nekopa wrote: post #37
omouse wrote: post #31

Earlier quoted context omitted.

First...IANAL (I Am Not A Lawyer) You aren't the only one, but with Freenet it's fully encrypted. Let's say you had a Freenet Silk Road application. You won't know it's a Silk Road web page that's being saved along with images of marijuana to your computer unless you go through an indexer/search site and even then you still won't know that those bits of data are stored specifically on your machine. So in order for th…

Not talking about the legal aspects, but, (and I really, really hate to bring up the "think about the children!" argument here) what about if I am unknowingly helping people who create and share child porn? It doesn't matter (to me) if I am on the hook for it or not, I just don't know (ethically?) how I would feel if I knew that was going on via my PC. Drugs I don't give a shit about, and I hate how the "think about…

The burden of supporting free speech is supporting speech that you may disagree with. Reputable civil liberties organizations like the ACLU regularly fight to protect the rights of groups (KKK[0], neo-Nazis[1], etc.) that they disagree with. You do not have to support the content of someone's speech to support their right to free speech.

[0] https://www.aclu.org/news/aclu-em-defends-kkks-right-free-sp...

[1] https://www.aclu.org/aclu-history-taking-stand-free-speech-s...

Post reply on HN