Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Caching the uncacheable: CloudFlare's Railgun

blog.cloudflare.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: Caching the uncacheable: CloudFlare's Railgun

#21

Post by nkurz »

jgrahamc wrote: post #16
nkurz wrote: post #15

Earlier quoted context omitted.

I don't think there's any push involved: Each end of the Railgun link keeps track of the last version of a web page that's been requested. When a new request comes in for a page that Railgun has already seen, only the changes are sent across the link. The listener component make an HTTP request to the real, origin web server for the uncacheable page, makes a comparison with the stored version and sends across the dif…

The numbers I'm reporting here are for the speed of download across Railgun and across HTTP between the origin server and a CloudFlare data center. Not the final download time to the end user. The issue with the end user number is deciding on what to report. We are currently rolling out a very large system for monitoring timing throughout our network and will be able (later this year) to report on actual end user tim…

Not the final download time to the end user.

OK. Still a useful measure, but a less dramatic one.

The issue with the end user number is deciding on what to report.

I would think that time to show a slightly changed page after a "refresh" or "reload" would be appropriate. What are the other choices?

For pages that are frequently accessed the deltas are often so small that they fit inside a single TCP packet, and because the connection between the two parts of Railgun is kept active problems with TCP connection time and slow start are eliminated.

Just reread this part, and not sure I understand it. Yes, there are no extra packets between the proxies, but in the base case there is only a single proxy and hence no extra connection time to consider. I'd think even a very fast proxy would introduce more latency than a hop on a backbone router. Or are you indeed pushing the per-user delta to the data center in anticipation of the request?

Re: Caching the uncacheable: CloudFlare's Railgun

#22

Post by bluelu »

So how does this work if you want to count the number of visitors, delivering customer specific versions of the site etc...? Does the bbc example even work because I can imagine that there multiple versions of the site depending on which IP is accessing BBC

Re: Caching the uncacheable: CloudFlare's Railgun

#23

Post by beaker52 »

rdl wrote: post #20
beaker52 wrote: post #12

Pushing 0.5kb opposed to 93kb to a downstream caching proxy isn't going to set the world on fire if you consider the data transfer speeds we typically achieve these days, especially if you also consider the processing overhead on both your network and the cache provider. They're not really caching the uncacheable, either.

If you're on the wrong side of a really long high latency link, it should make a big difference. Slow start, etc. Leaving a persistent connection between the cache and the server obviously does most of it (especially with latency mitigation tricks/tcp acceleration). It would be interesting to calculate the benefits of nothing vs. a cache with an accelerated persistent tcp connection vs. deltas. I suspect it's somethi…

The time to first byte latency due to long distance is going to be the same whether the payload is 2kb or 100kb.

If you have servers with a very low throughput outbound connection to the cache, then reducing the data transfered in this manner could be worth it. But 100kb over the wire at todays throughput is not going to add all that much latency to the whole transaction. As you suggested, some figures would be nice though.

Re: Caching the uncacheable: CloudFlare's Railgun

#25

Post by TazeTSchnitzel »

bluelu wrote: post #22

So how does this work if you want to count the number of visitors, delivering customer specific versions of the site etc...? Does the bbc example even work because I can imagine that there multiple versions of the site depending on which IP is accessing BBC

Well, one obvious thing with BBC News is that ads aren't shown for UK visitors but are for international visitors.

Re: Caching the uncacheable: CloudFlare's Railgun

#26

Post by rdl »

beaker52 wrote: post #23
rdl wrote: post #20

Earlier quoted context omitted.

If you're on the wrong side of a really long high latency link, it should make a big difference. Slow start, etc. Leaving a persistent connection between the cache and the server obviously does most of it (especially with latency mitigation tricks/tcp acceleration). It would be interesting to calculate the benefits of nothing vs. a cache with an accelerated persistent tcp connection vs. deltas. I suspect it's somethi…

The time to first byte latency due to long distance is going to be the same whether the payload is 2kb or 100kb. If you have servers with a very low throughput outbound connection to the cache, then reducing the data transfered in this manner could be worth it. But 100kb over the wire at todays throughput is not going to add all that much latency to the whole transaction. As you suggested, some figures would be nice…

First byte, yes, but won't you get the first 100 KB a lot faster due to being effectively 20ms away from it, vs. 320ms, thanks to slow start? Maybe little impact at 1KB, but for a 4MB abortion like some of the web pages I've seen, ...

(This all from TCP acceleration, not deltas, though. Deltas might give you 1ms on a 1G link if it saves you sending 100KB. BFD. Deltas to the edge, where you might be constrained by bandwidth on a mediocre 3G connection, is where deltas would rock - coupled with SPDY and tcp acceleration and caching and we'd be living in 2015.)

Re: Caching the uncacheable: CloudFlare's Railgun

#27

Post by marcusf »

jgrahamc wrote: post #14
reitzensteinm wrote: post #13

I'm having a hard time wrapping my head around the benefit of this - maybe John can elaborate some more. Taking the CNN front page for example. If you set a TTL of 60 seconds, and you have 14 edge locations (taken from the CloudFlare about page), you've got to satisfy 1440 * 14 = 20160 requests a day. The CNN page is currently 97527 bytes, which gzips down to 20,346 bytes. That's 391 megabytes per day. Serving the ed…

Firstly, we're not 'replacing HTTP'. HTTP runs over Railgun in the same way that it does over SPDY and the content servers are still running HTTP. The big benefit is not in terms of bandwidth saved (for us) it's in terms of total time to get the page. That's partly driven by latency and partly by bandwidth. Because we have worldwide data centers we can see high latency from say the data center in Miami and a web serv…

I'm still not sure how this solves the personalization use case. Usually (I think) the main problem is rendering the page, not pushing it down the wire. So if you still need to render for each request for a personalized page to make sure it's not been updated, where's the gain?

I think I might be slow, but I'm not sure where the wins are here and how you overcome the personalized rendering problem? I'd love for you to talk a bit about it.

Re: Caching the uncacheable: CloudFlare's Railgun

#28

Post by StavrosK »

marcusf wrote: post #27
jgrahamc wrote: post #14

Earlier quoted context omitted.

Firstly, we're not 'replacing HTTP'. HTTP runs over Railgun in the same way that it does over SPDY and the content servers are still running HTTP. The big benefit is not in terms of bandwidth saved (for us) it's in terms of total time to get the page. That's partly driven by latency and partly by bandwidth. Because we have worldwide data centers we can see high latency from say the data center in Miami and a web serv…

I'm still not sure how this solves the personalization use case. Usually (I think) the main problem is rendering the page, not pushing it down the wire. So if you still need to render for each request for a personalized page to make sure it's not been updated, where's the gain? I think I might be slow, but I'm not sure where the wins are here and how you overcome the personalized rendering problem? I'd love for you t…

It seems that all the gains are at the point between the upstream server and the CDN node. It has to send fewer packets for the entire page. However, I'm not sure this isn't pretty much as efficient as a large TCP window. Of course, I'm not the one whose job it is to worry about these things, so take this with a grain of salt.

Re: Caching the uncacheable: CloudFlare's Railgun

#29

Post by beaker52 »

rdl wrote: post #26
beaker52 wrote: post #23

Earlier quoted context omitted.

The time to first byte latency due to long distance is going to be the same whether the payload is 2kb or 100kb. If you have servers with a very low throughput outbound connection to the cache, then reducing the data transfered in this manner could be worth it. But 100kb over the wire at todays throughput is not going to add all that much latency to the whole transaction. As you suggested, some figures would be nice…

First byte, yes, but won't you get the first 100 KB a lot faster due to being effectively 20ms away from it, vs. 320ms, thanks to slow start? Maybe little impact at 1KB, but for a 4MB abortion like some of the web pages I've seen, ... (This all from TCP acceleration, not deltas, though. Deltas might give you 1ms on a 1G link if it saves you sending 100KB. BFD. Deltas to the edge , where you might be constrained by ba…

Remember that we're talking about content provider to content cache here. Server infrastructure to server infrastructure, usually connected by huge pipes.

Passing deltas to a client over a mobile data network would be an awesome development, I agree. With some additional specification to HTTP and vendor implementation on clients, it'd definitely be possible.

Re: Caching the uncacheable: CloudFlare's Railgun

#30

Post by StavrosK »

bluelu wrote: post #22

So how does this work if you want to count the number of visitors, delivering customer specific versions of the site etc...? Does the bbc example even work because I can imagine that there multiple versions of the site depending on which IP is accessing BBC

I'm guessing the visitor IP still gets passed. This looks to be entirely synchronous, so everything would still work. The CDN just only has to fetch a delta of the page rather than the whole.
Post reply on HN