Live data from Hacker News

Reweaving the web: A slew of startups is trying to decentralise the online world

economist.com

41–50 of 72 posts

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#41
post #40
post #36

Earlier quoted context omitted.

> despite popular believe P2P networks suffer from centralization issues in P2P bootstrapping. That's a fairly modest problem. You need a bunch of nodes with above-average capacity to handle the bootstrap traffic and a way to occasionally update lists of such nodes. But a bunch of 10$ per quarter virtual boxes at a hoster of your choice and a bunch of domains with DNS SRV records can do that job. In the absence of an…

The number of existing bootstrapping nodes do not solve the fundamental problem of current centralized bootstrapping. Not even if a virtual box costs $1 instead of $10. The issue is how a connecting node knows about other nodes in the network, and the current propagation of this information i.e. the DNS or IP of the seed nodes introduces centralization. The closest and best solution for this problem was the early day…

It is decentralized in the sense that it is not centralized, i.e. there is no central entity in control managing entry into the network.

The key point is that any node in a p2p network can be used to join the network, so any number of independent entities can publish lists of ways to join the network.

Certainly, i would also love some multicast/anycast support on the IP level, but we don't have that, so making sure that you have many independent entry points into the network is the next best thing.

Maybe you're thinking distributed, not decentralized? https://i.imgur.com/YsiiKeq.png

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#42

I wish the article had spent more time discussing the root causes of the centralized Internet and ways to address them. This is the most interesting problem in tech right now in my opinion and many people are working on it from different angles but I've yet to see anything which really struck me as a killer app for the "re-decentralized" web. We have to look at why these centralized services have succeeded: for insta…

>> as a killer app for the "re-decentralized" web. IPv6 anycast looks quite promising[0][1][2] to help solve the problem with fully decentralizing bootstrapping of P2P nodes. This is a central issue currently affecting all p2p like networks including Ethereum, BitTorrent, Gnutella, Bitcoin, ... main issue is probably lack of IPv6 anycast support (??) due to misinformation, broken middle boxes, and and lack of underst…

> main issue is probably lack of IPv6 anycast support (??)

It requires root or user-namespace shenanigans to assign the v6 address needed by the p2p client.

And access to BGP tables.

And they don't estimate how long such a bootstrap process would take, since for small networks it would potentially have to scan millions of candidate addresses.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#43
post #31
post #11

It's cyclical and this should be obvious to anyone in technology. We move between eras of centralisation and decentralisation. The internet started as a decentralised network in which we then built high capacity centralised systems at an accelerated rate. We're now about to shift back into a new era of decentralisation fuelled by edge/fog computing, IoT and p2p networks all on top of the existing infrastructure. The…

Or consider the much more "cynical" (and in my mind more obvious, if it wasn't for our habit of seeing patterns everywhere) view that there might not be any structure, cyclical or other, to the course of history. In that view, whether the future of the internet will be centralized or decentralized is undetermined and will most likely be a mishmash of both, the extent of with will depend on inertia, ideology, economic…

I think the tendency will be towards centralization because centralization is more efficient.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#44
post #40
post #36

Earlier quoted context omitted.

> despite popular believe P2P networks suffer from centralization issues in P2P bootstrapping. That's a fairly modest problem. You need a bunch of nodes with above-average capacity to handle the bootstrap traffic and a way to occasionally update lists of such nodes. But a bunch of 10$ per quarter virtual boxes at a hoster of your choice and a bunch of domains with DNS SRV records can do that job. In the absence of an…

The number of existing bootstrapping nodes do not solve the fundamental problem of current centralized bootstrapping. Not even if a virtual box costs $1 instead of $10. The issue is how a connecting node knows about other nodes in the network, and the current propagation of this information i.e. the DNS or IP of the seed nodes introduces centralization. The closest and best solution for this problem was the early day…

I don't think bootstrapping per se is a problem. You only have to do it on first install and this allows for a simple solution to distribute bootstrapping in time, i.e. changing hardcoded bootstrapping data every N minutes, so new users could bootstrap from different nodes, than the users before them and so on.

A bigger problem is how software is developed. The development itself is centralized, but even worse, traditional software development approaches cannot predict all future problems, but rely on fixing them to keep the software working, which means it must be delivered to users in a centralized way too.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#45
post #19

Earlier quoted context omitted.

This probably won't be a popular opinion, but it seems like Google/Facebook/etc are entrenched too deeply in this incarnation of the web to be displaced any time soon. They seem to understand where they are too - it goes beyond just buying out competitors. Look at Facebook's Internet Lite plan for India.

At one point IBM was deeply entrenched in the industry. At another point in time it was Microsoft. This too, shall pass.

At the time, computing didn't have the market penetration it has today. In other words, even if IBM and Microsoft had 99.9% of the market (which they had not), it didn't matter because only a tiny part of the population had computers at home and even less so in their pockets. Today's internet penetration is way bigger, almost anyone at least in the West has either a phone, laptop, tablet or desktop with a way to access Google and Facebook. Won't just go away as easily.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#46
post #4
post #3

The narrative for the last few years, at least in the mainstream media, is that Google Facebook Twitter Microsoft et al. are too big for anyone else to replace them. It's the old "inevitability" argument - if your new thing gets too big, they'll just buy you up. Let's hope people continue to try to create services that a big percentage of people use and then resist to sell to the big data companies.

I loved the web in the early mid-2000s. Web 2.0 so many startups, so many ideas, so many failures. It was great. Lots more hope than what is out there right now.

It reminds me of early 20th century footage of barnstorming aircraft inventors. Now it's just Boeing and Airbus and precious little in the way of real innovation.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#47
post #31

Earlier quoted context omitted.

Or consider the much more "cynical" (and in my mind more obvious, if it wasn't for our habit of seeing patterns everywhere) view that there might not be any structure, cyclical or other, to the course of history. In that view, whether the future of the internet will be centralized or decentralized is undetermined and will most likely be a mishmash of both, the extent of with will depend on inertia, ideology, economic…

I think the tendency will be towards centralization because centralization is more efficient.

Not for transferring large files. A centralized architecture is the most expensive, slow way to do that.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#48
post #44
post #40

Earlier quoted context omitted.

The number of existing bootstrapping nodes do not solve the fundamental problem of current centralized bootstrapping. Not even if a virtual box costs $1 instead of $10. The issue is how a connecting node knows about other nodes in the network, and the current propagation of this information i.e. the DNS or IP of the seed nodes introduces centralization. The closest and best solution for this problem was the early day…

I don't think bootstrapping per se is a problem. You only have to do it on first install and this allows for a simple solution to distribute bootstrapping in time, i.e. changing hardcoded bootstrapping data every N minutes, so new users could bootstrap from different nodes, than the users before them and so on. A bigger problem is how software is developed. The development itself is centralized, but even worse, tradi…

I am sorry, but centralized network bootstrapping is a very well known issue and unsolved problem of decentralized networks.

http://ryandoyle.net/assets/papers/Distributed_Bootstrapping...

http://grothoff.org/christian/dasp2p.pdf

http://www.net.in.tum.de/fileadmin/TUM/NET/NET-2014-08-1/NET...

You say: "i.e. changing hardcoded bootstrapping data every N minutes, so new users could bootstrap from different nodes"

The fundamental issue is, changing the data where? The node who wants to connect to the peer network, obviously cannot obtain the information from the peer network itself (as node is not connected), so obtain the information from where? Currently, all decentralized applications, including my system Streembit use the techniques of obtaining the information from a centralized source - which is the oxymoron of decentralization. If a web services or other centralized applications provide the new node with the list of existing listening/connected nodes then the solution is surely not decentralized. Government agencies or cyber-criminals only need to attack the hard coded, listening seed nodes and then the network is done and never can be back again until a new list of hardcoded seed nodes is published via the application source code or via other channels.

As I said above, Satoshi's original idea to obtain the seed info from IRC was the closest to decentralization, but since IRC is centralized itself I am sure you can see how far that is from the a decentralized bootstrapping.

We have the solutions on local decentralized networks such as mDNS and UDP multicasting which uses protocol level solutions, and we are investigating to solve the problem at Streembit with IPv6 anycasting.

>>

I disagree. Open source software can be forked and then you can adopt as much democratic development methods and governance as you want.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#49
post #48
post #44

Earlier quoted context omitted.

I don't think bootstrapping per se is a problem. You only have to do it on first install and this allows for a simple solution to distribute bootstrapping in time, i.e. changing hardcoded bootstrapping data every N minutes, so new users could bootstrap from different nodes, than the users before them and so on. A bigger problem is how software is developed. The development itself is centralized, but even worse, tradi…

I am sorry, but centralized network bootstrapping is a very well known issue and unsolved problem of decentralized networks. http://ryandoyle.net/assets/papers/Distributed_Bootstrapping... http://grothoff.org/christian/dasp2p.pdf http://www.net.in.tum.de/fileadmin/TUM/NET/NET-2014-08-1/NET... You say: "i.e. changing hardcoded bootstrapping data every N minutes, so new users could bootstrap from different nodes" The f…

> The fundamental issue is, changing the data where? The node who wants to connect to the peer network, obviously cannot obtain the information from the peer network itself (as node is not connected), so obtain the information from where?

Ok, you are assuming that the node already has the binary somehow. But that's not the case. In the real world we have to ship binaries to users. And this is where you can put your different bootstrapping data for different users.

You can go a lot farther: let users generate a binary distribution of the software to share with each other and hardcode bootstrapping data there obtained from a running network by that user.

> Open source software can be forked

The majority of users are not going to do that, they just don't have the skills. At best there will be a few popular distributions of the same "decentralized" software, with majority of installations controlled by a few entities. At worst - just one centralized entity that controls every installation.

Re: Reweaving the web: A slew of startups is trying to decentralise the online world

#50
post #33

All these articles in the mainstream media about decentralized computing are very nice and the Ethereum and Bitcoin PR teams are doing a fine job to disseminate the idea of decentralized computing. At the same time, I think we need to be more honest about one of the fundamental issues of decentralized networking, which is that the bootstrapping of the network is not decentralized. The bootstrapping in fact centralize…

This is true and IMO does not get enough discussion.

It's a relatively easy problem to solve, but I'm not sure that hardcoding IP addresses into programs is the best solution for users who do not compile from source.

Out of curiosity I once reviewed the source code for a certain "onion router" program and found a handful of hardcoded IP addresses. Servers at MIT if I recall correctly. If these servers are inaccessible, what is the effect?

Perhaps "bootstrapping" is simply a matter of voluntarily downloading a trusted file, containing some reachable IP addresses.

Can millions of people all download the same file? If no, then how does Firefox, Adobe Flash or Chrome get installed on so many computers?

I consider the web (cf. internet) as more or less "bootstrapped". This is because of the preference for domain names versus IP addresses. It necessitates lookups.

Bootstrapping all starts with one file: root.zone.

Since 1993, this file has not changed very often. The IP address for the server where one can download it does not change very often either.

Once I have that file, whether the root servers are working is irrelevant. Don't need them. Only need TLD nameservers, .com, etc. to be working. And of course one can download a local copy of the .com zone, etc., as a backup.

But assuming those root servers are working I do not even need this file. Long ago I memorized the IP addresses for one root server and one com nameserver.

As long as I have a means to send DNS packets[1] I can look up the IP address for any domain name and I can "surf the web". Or at least get myself out of a jam without the use of recursive DNS servers.

1. Even netcat will work, as explained in its tutorials. I do not use "dig" or BIND libraries. Nor do I ever need to set the recursive bit. I can do the resolution using non-recursive requests, using some helper scripts/programs I wrote. Interestingly, using this method beats the speed of a cold cache in nearly all cases.

For the newly decentralized web, we will need supernodes that do not pass traffic but only provide port information to allow users to connect with each other through NAT.

The IP addresses of these supernodes will be the "bootstrap". There could be millions of supernodes.

Today, the "web" is addresses of servers controlled by companies pandering to advertisers.

The "decentralized web" is addresses of users that can connect with each other to form communities. Those communities can be connected with each other, and so on.

The "content" being served to users by today's web is largely "user-generated". But overrun with advertising and "analytics". The content is served indiscriminantly to the open internet. And even if there are access controls, because the data is centralized with a handful of companies it's an easy target for third parties who want access to it.

Tomorrow's decentralized web allows user to selectively exchange their content with each other, without the companies and advertisers in the middle. Third parties wanting access may have to be more clever and more selective.

Post reply on HN