Live data from Hacker News

Peerwiki: all of Wikipedia on BitTorrent

github.com

21–30 of 31 posts

Re: Peerwiki: all of Wikipedia on BitTorrent

#21
post #10

Earlier quoted context omitted.

Closed source?

I think so, yes.

I wouldn't want something like this to be closed source. It's very important that the security of the decentralization can be validated.

And in these NSA times it's even more important.

Re: Peerwiki: all of Wikipedia on BitTorrent

#22

Would a blockchain-like solution work here? Where new edits piled up on top of the existing data, constantly sharing it across all servers?

It could but it would be an unnecessary hurdle, because there is no need for a global consensus on a single version of the encyclopedia. Think of it as a git tree, and checkout the branch you like. With a currency, it's imperative that everyone refers to the same branch all the time, not so with an encyclopedia. In practice there would be a few "popular" branches, and one would likely dominate, so that it would be tr…

It's not imperative that everyone refer to the same branch all the time for currency either. If User A gives Bitcoin to User B, and User C gives Bitcoin to User D, there's no need for them to be on the same branch as the two transactions are unrelated.

It's when you need to merge branches back together that things tend to get messy - with currency or an encyclopedia, and everyone being on the same branch all the time is better, for both cases.

I can just imagine what the 'popular' branch for an encyclopedia would look like if 4chan set their sites on causing problems for it.

Re: Peerwiki: all of Wikipedia on BitTorrent

#23

Would a blockchain-like solution work here? Where new edits piled up on top of the existing data, constantly sharing it across all servers?

It could but it would be an unnecessary hurdle, because there is no need for a global consensus on a single version of the encyclopedia. Think of it as a git tree, and checkout the branch you like. With a currency, it's imperative that everyone refers to the same branch all the time, not so with an encyclopedia. In practice there would be a few "popular" branches, and one would likely dominate, so that it would be tr…

If you consider malicious actors (http://en.wikipedia.org/wiki/Wikipedia:Long-term_abuse), some form of local consensus becomes necessary. In the real Wikipedia this is handled by admin actions right now. And that's really the only workable system I can come up with right now.

You could have different branches with different admin teams, but in a sense we have that technology already: anybody can download an XML dump of the Wikipedia database and set up their own Wikipedia clone with minimal effort.

Re: Peerwiki: all of Wikipedia on BitTorrent

#24

Earlier quoted context omitted.

It could but it would be an unnecessary hurdle, because there is no need for a global consensus on a single version of the encyclopedia. Think of it as a git tree, and checkout the branch you like. With a currency, it's imperative that everyone refers to the same branch all the time, not so with an encyclopedia. In practice there would be a few "popular" branches, and one would likely dominate, so that it would be tr…

It's not imperative that everyone refer to the same branch all the time for currency either. If User A gives Bitcoin to User B, and User C gives Bitcoin to User D, there's no need for them to be on the same branch as the two transactions are unrelated. It's when you need to merge branches back together that things tend to get messy - with currency or an encyclopedia, and everyone being on the same branch all the time…

1) Sure, some operations are commutative and compatible, and you can import them from one branch to another, but in the general case, they aren't

2) Same thing that happens currently on Wikipedia. There's a set of rules that determine what edits are valid and which aren't, and a local client can applies those rules (or not) to determine the HEAD version. There's nothing that prevents a Sybil attack on Wikipedia, but there doesn't seem to be the need for it.

Re: Peerwiki: all of Wikipedia on BitTorrent

#25
post #21

Earlier quoted context omitted.

I think so, yes.

I wouldn't want something like this to be closed source. It's very important that the security of the decentralization can be validated. And in these NSA times it's even more important.

If the concept works well, an open source variant is sure to be created precisely for this reason.

Re: Peerwiki: all of Wikipedia on BitTorrent

#26

Earlier quoted context omitted.

It could but it would be an unnecessary hurdle, because there is no need for a global consensus on a single version of the encyclopedia. Think of it as a git tree, and checkout the branch you like. With a currency, it's imperative that everyone refers to the same branch all the time, not so with an encyclopedia. In practice there would be a few "popular" branches, and one would likely dominate, so that it would be tr…

If you consider malicious actors ( http://en.wikipedia.org/wiki/Wikipedia:Long-term_abuse ), some form of local consensus becomes necessary. In the real Wikipedia this is handled by admin actions right now. And that's really the only workable system I can come up with right now. You could have different branches with different admin teams, but in a sense we have that technology already: anybody can download an XML du…

Yes, we do have that technology, yes it's Wikipedia, yes it can be distributed, no that doesn't require a blockchain nor particularly benefit from one.

Re: Peerwiki: all of Wikipedia on BitTorrent

#27

Author here. If anyone is interested I presented this at jsconf.eu last year as part of a BitTorrent talk, https://www.youtube.com/watch?v=BTCsSwCpGP8 (slides: http://mafintosh.github.io/slides/jsconf-2014/jsconf-eu-2014... )

What are the obstacles to running this client-side in the browser? That would lower the barrier to entry for users by a lot.

Edit: I just finished watching your talk where you mention that https://github.com/feross/webtorrent is already doing that.

Re: Peerwiki: all of Wikipedia on BitTorrent

#28
post #11
post #3

Earlier quoted context omitted.

Pretty sure that it's: When you want to go on a wikipedia article, it requests the file from other peers in the network using the bittorrent protocol. It's a copy of Wikipedia placed on bittorrent, presumably with some semantics for article updates.

It looks like it uses a single Wikipedia dump from over a year ago. Would be cool if it supported deltas somehow, so the network doesn't split when a new dump is used.

this breaks the bittorrent.

Re: Peerwiki: all of Wikipedia on BitTorrent

#29
post #11
post #3

Earlier quoted context omitted.

Pretty sure that it's: When you want to go on a wikipedia article, it requests the file from other peers in the network using the bittorrent protocol. It's a copy of Wikipedia placed on bittorrent, presumably with some semantics for article updates.

It looks like it uses a single Wikipedia dump from over a year ago. Would be cool if it supported deltas somehow, so the network doesn't split when a new dump is used.

Bittorrent doesn't have support for that, but you could publish torrents that just host a full version of all the modified pages.

Re: Peerwiki: all of Wikipedia on BitTorrent

#30
post #11
post #3

Earlier quoted context omitted.

Pretty sure that it's: When you want to go on a wikipedia article, it requests the file from other peers in the network using the bittorrent protocol. It's a copy of Wikipedia placed on bittorrent, presumably with some semantics for article updates.

It looks like it uses a single Wikipedia dump from over a year ago. Would be cool if it supported deltas somehow, so the network doesn't split when a new dump is used.

Found out recently that there's a cool project underway to make a distributed wiki - http://wardcunningham.github.io/.
Post reply on HN