Inspired by this, can anyone explain why distributed protocols more often opt for centralized consensus algorithms like Raft, instead of decentralized schemes like Chord or Kademlia? In all cases, the underlying data structure is a shared key/value store. Intuitively, the p2p approach feels more robust, since each node only needs to worry about itself, and every node is the same. So why add the coordinator node? Is it still the right choice in $current_year, even after so many hours of development invested into strong p2p consensus protocols like libp2p powering Ethereum (currently ~11k nodes btw, not actually that big – and many operated by small number of entities)?
Static torrent website with peer-to-peer queries over BitTorrent on 2M records
31–40 of 110 posts
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#32a decade ago, I had the idea of a distributed internet wide filesystem where the chunks could be duplicated over and over again over the internet. Something survivable and loosely/eventually consistent when updated.
Someone appears to have built at least part of it.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#33Yup. Calling it. This is the future, may not seem like it to everyone but this is a part of actually new and extremely useful, passively scalable technology. Imagine (good) unkillable zombie databases- so long as the name of a piece of data is known (its hash).. someone, somewhere, might make it possible for you to answer your query, without even needing to setup a server setup.. that’s _it_! Not to mention it might…
So this is the true Web3 then?
I know everyone is fed up with the cliche of web3, but having done some side projects with web3 (as in npm i web3) and ipfs, the posted article here is very exciting.
This posted link could be another piece of the jigsaw (Solidity, IPFS, IPNS, ...) that I think will come together to make interesting apps in the future. Solidity doesn't mean necessarily spending $100 to make a function call - there are other chains, off chain stuff being developed, and you could host a private chain for your app.
While none of this stuff can do something new you can't do with Postgres - you can create more open and perhaps 'honest' applications where everyone can see the data-engine and understand it. So it is more of a cultural shift. For example, if you make a Twitter this way, you don't need to rely on an API. The data is there for everyone, all of it.
For example, take Uniswap. It's not a company like Facebook, it's an open protocol. Swapping tokens is now functionally open source because of that. There is no "Facebook of swapping tokens". And this can be used as a building block for other apps. Not necessarily just "gambling" or "trading" either.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#34Since ipfs is hash addressable, is there a decentralized way to point to the latest hash of the content. I can point a domain to it, but domain can be taken down.
ENS [https://ens.domains/] names can point to IPFS addresses. https://jamescarnley.eth (can use https://jamescarnley.eth.link if your browser doesn't support IPFS) is my IPFS powered website that can't be taken down by anyone.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#35Earlier quoted context omitted.
So this is the true Web3 then?
I know everyone is fed up with the cliche of web3, but having done some side projects with web3 (as in npm i web3) and ipfs, the posted article here is very exciting. This posted link could be another piece of the jigsaw (Solidity, IPFS, IPNS, ...) that I think will come together to make interesting apps in the future. Solidity doesn't mean necessarily spending $100 to make a function call - there are other chains, o…
What you’re describing is so 101 level, I don’t mean it in an insulting way but in bewilderment from the typical criticisms from within tech from which you’d think what you’re describing is impossible mythology or completely off the mark
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#36Yup. Calling it. This is the future, may not seem like it to everyone but this is a part of actually new and extremely useful, passively scalable technology. Imagine (good) unkillable zombie databases- so long as the name of a piece of data is known (its hash).. someone, somewhere, might make it possible for you to answer your query, without even needing to setup a server setup.. that’s _it_! Not to mention it might…
Unfortunately this is still killable. Someone needs to periodically publish up to date databases using some outside protocol. Also if you participate in the public DHT, then your participation is public and can have consequences on your real-life legal entity. If you want to use TOR and onion sites, I don't think this really adds anything to those. I think this just helps you optionaly crowdsource bandwidth.
With a simple protocol on top it can be quite resilient. The database is updated only by a group of 3 admins, who perform the update once a week. When one admin initiates an update, the other two ask him the passphrase. If the answer is "tomato" (means "I'm compromised"), the update is rejected and they find another admin. Regular users are generally safe: the laws dont punish downloaders, only uploaders. You can expand the admin team to 5 members, so RIAA (or whoever stalks torrent sites) would have to compromise 2 admins simultaneously. Don't forget that the admins can be in different countries, making their legal pursuit a nightmare.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#37This is cool but if I understand it right, it's actually WebTorrent protocol via WebRTC, not inter-torrentable of vanilla BitTorrents out there?
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#38This is cool but if I understand it right, it's actually WebTorrent protocol via WebRTC, not inter-torrentable of vanilla BitTorrents out there?
Webtorrent support has been added to Vuze and libtorrent (backs clients like deluge and qBittorrent), so there is some ability for mainstream swarms to interact with webtorrent peers.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#39Can someone explain this pro: "Database size and number of records are not a problem"
How is this the case if earlier there is a "limitation of not being able to ask ONLY for pieces I'm interested in"?
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#40Yup. Calling it. This is the future, may not seem like it to everyone but this is a part of actually new and extremely useful, passively scalable technology. Imagine (good) unkillable zombie databases- so long as the name of a piece of data is known (its hash).. someone, somewhere, might make it possible for you to answer your query, without even needing to setup a server setup.. that’s _it_! Not to mention it might…
I've built something akin to this, but with the idea of applications distributing API`s where the databases are also torrented but working behind the api's, so that developers can build basically anything.
In my case i've implemented a new "browser" based on Chrome that allows this to work, without having to resort to browser-only infrastructure (for instance applications can dodge Javascript and also call RPC api's from other applications directly).
The applications and the applications data are distributed over torrent and managed to work together in the same environment as a flock where one app can consume its own apis and also the api's from others.
service Search {
rpc doQuery(string query) => (array result) // the access to the sqlite db from torrents is encapsulated here
}
---