Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Uber-like CDN

medium.com

Post reply on HN
Print view View original

51–60 of 86 posts

Re: An Uber-like CDN

#51

Post by jsnell »

paxys wrote: post #35
jsnell wrote: post #7

There's a bunch of really obvious objections to this idea around performance, privacy, economics, UX, and reliability. Impressively, the post did not manage to address any of them up front. 1. Performance. In terms of latency, this is adding an extra round-trip to each page request (to get the metadata). It's also likely going to load different resources from different servers (hostname, IP) preventing connection reu…

1 is something every CDN in the world already deals with. The metadata system ultimately needs to be its own little CDN, and can achieve the same level of performance (e.g. by deploying at the edge) just without the bandwidth costs. 2 is definitely an issue. 3 is a maybe . There's definitely a model where people with access to cheap servers and bandwidth can get on board and not have to worry about the application-le…

1. None of the issues I listed affect traditional CDNs though.

3. Why would the people running a peer have cheaper access to servers and bandwidth than a company specializing in that business? Everything about this business should have huge economies of scale (buying hardware, system administration, transit/peering).

The only situation where I can see a distributed model winning out is if the entity running the peer does not pay for resources but gets them with fraud or theft, e.g. paying for cloud servers with stolen credit cards.

Re: An Uber-like CDN

#54

Post by jefftk »

This says it's a system for delivering content "efficiently", but it doesn't look like this manages that. Normally loading a resource is a round trip to your server, which might be a bit inefficient because your server could be on the other side of the world. But what they replace it with is:

1. Round trip to Farba to get metadata.

2. Round trip to your "driver" to get the resource.

3. Computing the hash of the resource to ensure the "driver" didn't cheat.

It could still be worth it as a way of saving money on bandwidth, but I'd be quite surprised if you got the speed benefits of a CDN.

Re: An Uber-like CDN

#55

Post by bastawhiz »

mranton wrote: post #16
amalcon wrote: post #4

Something doesn't add up here. They're actively telling third parties to lease cloud servers with large amounts of free bandwidth, and allocate them to this purpose. I see four possibilities: - They are paying the third party less than what it costs to rent the server, in which case the third party is losing money and shouldn't be doing this. - They are paying the third party at least what it costs to rent the server…

>Something doesn't add up here. They're actively telling third parties to lease cloud servers with large amounts of free bandwidth: Bare-metal servers are much better is our case. Please note, bandwidth is not free. You can follow the links and see the actual price for it. If LeaseWeb tells you: 1 Gbps dedicated/unlimited per $143/month, it means you can use it 24x7. >- They are paying the third party less than what…

> That's why the _Peer_ sets the price for his service.

But that breaks the economics of this, no? If you're in the US and charge $6/TB and I'm in Germany and charge $100/TB, why would traffic ever go to my server? If your US server gets overloaded, am I now making $100/TB? How does the network ensure that this is both fair and competitive? Moreover, how can a customer know that they're not going to get a surprise bill for an order of magnitude more than they planned for?

Re: An Uber-like CDN

#56

Post by bastawhiz »

What stops a peer from, say, injecting malicious JavaScript into pages? Or going rogue and serving porn for every request for a JPEG? What if they only do it for every 1000th request?

You could send a hash of the file to the client from the coordination server and check it. If the hash isn't valid, you can re-request the file from another peer. But I don't think you can trust the client to report abuse: a malicious client could report innocent peers (e.g., a vindictive Chrome extension).

Or, the site owner could manipulate the coordination JavaScript to report the downloaded file as invalid/corrupt/tampered with, and then just not re-request the file (and there's nothing you can do to stop them). So in that case, the site owner is managing to not pay for the service (assuming you don't bill for downloads that were reported as invalid).

But on top of all of that, this is _slow_. It's slow on my (quite fast) internet. If you're using this, it's because you're trying to get cheap bandwidth, not because you value performance. The whole point of a proper CDN is to improve performance by spreading out load and moving the edge server physically closer to the user. This might be putting the edge server physically closer to the user, but the mechanism to do that seems to take almost as long as the request/download of the asset itself.

Re: An Uber-like CDN

#57

Post by bastawhiz »

mranton wrote: post #27
uncertainrhymes wrote: post #13

There are several red flags here, not the least of which is that a CDN isn't just about bandwidth, it is about disk . If a cache node doesn't have the object, it has to go to origin to get it. That has a cost at origin, which most CDN customers want to minimize. Cache management (eviction, invalidation, etc) is done where? By the look of that diagram, you are actually doubling your origin traffic just to push to the…

>a CDN isn't just about bandwidth, it is about _disk_ The disk performance - is the weakest part of the Peer. It is a part you can't guarantee any QoS on a cheap server. The idea is to keep popular cache on Peers in memory. Everything else we will host from a limited number of regular servers (belong to Farba), with high-performance disk subsystem. >Cache management (eviction, invalidation, etc) is done where? yes. Y…

> The disk performance - is the weakest part of the Peer. It is a part you can't guarantee any QoS on a cheap server. The idea is to keep popular cache on Peers in memory. Everything else we will host from a limited number of regular servers (belong to Farba), with high-performance disk subsystem.

I use a CDN to serve about 50GB of files stored on my origin, which turn over every week or so (a new 50GB every week). What you're saying means that _at best_ I need at least 3-4 peers in order to keep one copy of all of my files available. That's your using your numbers for a LeaseWeb or OneProvider peer. And if I'm competing with other customers, my content is getting evicted, resulting in more hits to my origin. If I'm actually using your system as a CDN where peers should be physically close to my users, that means that I'm filling up RAM of lots and lots of peers just so that enough peers that are close to my users can serve the files.

Re: An Uber-like CDN

#58

Post by vineyardmike »

mranton wrote: post #14
jsnell wrote: post #7

There's a bunch of really obvious objections to this idea around performance, privacy, economics, UX, and reliability. Impressively, the post did not manage to address any of them up front. 1. Performance. In terms of latency, this is adding an extra round-trip to each page request (to get the metadata). It's also likely going to load different resources from different servers (hostname, IP) preventing connection reu…

>1. Performance. In terms of latency, this is adding an extra round-trip to each page request (to get the metadata). It's also likely going to load different resources from different servers (hostname, IP) preventing connection reuse. I guess they can't even start fetching data until the full page has loaded? fair point. We use same technique as lazy-load (you have to wait the full page has loaded). >3. Economics. If…

> This is how Uber-economy works.

> We just don't want to be another CDN provider.

Don't do something differently for the sake of being different. Do something differently because it unlocks some new capability.

Uber was special for a few reasons. Primarily, it was a better UX to use an app for a taxi, and apps couldn't exist when taxis were first popularized. Secondarily, Uber not driving taxis themselves was "different" in labor law handling. Uber didn't own cars/employ drivers not because "they didn't want to be another taxi company" but because they wanted to unlock the capability to pay people differently. It's the same story for AirBnB - what's legal for rental units is not the same as what's legal for hotel rooms.

Any company can rent or own thousands of servers across the planet, we know this because CDNs do it today, and because if an individual can do it, so can a company.

Re: An Uber-like CDN

#59

Post by zamnos »

jsnell wrote: post #51
paxys wrote: post #35

Earlier quoted context omitted.

1 is something every CDN in the world already deals with. The metadata system ultimately needs to be its own little CDN, and can achieve the same level of performance (e.g. by deploying at the edge) just without the bandwidth costs. 2 is definitely an issue. 3 is a maybe . There's definitely a model where people with access to cheap servers and bandwidth can get on board and not have to worry about the application-le…

1. None of the issues I listed affect traditional CDNs though. 3. Why would the people running a peer have cheaper access to servers and bandwidth than a company specializing in that business? Everything about this business should have huge economies of scale (buying hardware, system administration, transit/peering). The only situation where I can see a distributed model winning out is if the entity running the peer…

It's a matter of locale. Having an entrepreneur in South Africa or Tibet hear about the money making opportunity, setting up a peer, and then paying them for it is better than flying there for a month to set up a business presence and figuring out all the local everythings. Akamai et al has already done this, and charge money commensurate with having done that. This group seems to be banking on someone in Kathmandu hearing about this opportunity and setting up a peer there.

Re: An Uber-like CDN

#60

Post by bastawhiz »

mranton wrote: post #44
uncertainrhymes wrote: post #38

Earlier quoted context omitted.

Your example servers have 16-32 GB of memory, which is basically nothing. If you ever ramp to real world traffic the peer will be constantly evicting objects (if you use a normal LRU). Even if you backstop that with a mid-tier belonging to Farba to protect the origin, what kind of cache hit rate are you expecting at the peer?

>Your example servers have 16-32 GB of memory, which is basically nothing we just need to add more Peers and evenly distribute the workload. > what kind of cache hit rate are you expecting at the peer? I don't know. We have too many moving parts right now and can't predict anything.

> we just need to add more Peers and evenly distribute the workload.

That doesn't make sense! If you add more peers just to keep more files in memory, each peer is serving less and less data. There's a fixed, finite amount of usage that the system will receive, and the more peers you add, the more that usage gets subdivided among the peers. Which means they make less money, which makes the system unprofitable. Moreover, you're spending more money to distribute the file to more peers (well, you're not, you're offloading that to the customer right now), which means the more peers, the more expensive for you or the customer.

The problem you have is that handling more files means cutting peer profits without adding any benefit for yourself or the peers. Storage isn't factored into pricing, which means that any accommodation you make to increase availability (an important part of being a CDN!), the worse the economics end up being.

Post reply on HN