Caching the uncacheable: CloudFlare's Railgun

I meant in terms of server-side caching.

I'm having a hard time wrapping my head around the benefit of this - maybe John can elaborate some more.

Taking the CNN front page for example. If you set a TTL of 60 seconds, and you have 14 edge locations (taken from the CloudFlare about page), you've got to satisfy 1440 * 14 = 20160 requests a day. The CNN page is currently 97527 bytes, which gzips down to 20,346 bytes. That's 391 megabytes per day. Serving the edge locations even with this relatively short TTL is trivial.

Now, the TCP connection means the content can be pushed and not pulled, so latency is better. It also means that caching a lot of pages will become cheaper (though still expensive).

But it doesn't seem like a lot of benefit for essentially replacing HTTP (between the content servers and the edge nodes), for all the proprietary software and vendor lockin that entails. For each byte you're sending to edge nodes, it's going to be served up orders of magnitude more from the edge nodes to end users, which seems like where almost all the cost would lay.

I'm sure they know what they're doing, and that their customers have asked for this, but think some real world case studies would help make it click. That the blog post is going on about 'caching the uncacheable' really doesn't help.

I don't think there's any push involved:

Each end of the Railgun link keeps track of the last version of a web page that's been requested. When a new request comes in for a page that Railgun has already seen, only the changes are sent across the link. The listener component make an HTTP request to the real, origin web server for the uncacheable page, makes a comparison with the stored version and sends across the differences. The sender then reconstructs the page from its cache and the difference sent by the other side.

I take this to mean that there are two chained proxies, each proxying pages on a per-user basis. Since the upstream server-side proxy knows what the downstream client-side proxy has cached, it can send a very efficient shorthand describing how the page has changed without having to resend the information that's already been sent.

I think it's a smart good approach. So long as the origin-proxy is inside the datacenter, it clearly would save a lot on data charges. But I'm surprised the speedup is as much as JGC reports since you still have to pass the full page over the last-mile to user. I would have thought that was the slowest link. Is the core internet so congested that this is not the case? I'm presuming the data center and the origin server have very good throughput, and that even a very short message would have the same latency.

And for those that care about this sort of thing: Railgun is about 4,000 lines of Go.

I've noticed that using bzip, cranked up to it's max level, is really effective for compressing dynamic HTML. I've managed to achieve what I suspect is comparable to Railgun by simply having this compression applied on a multiplexed TCP tunnel in rather large blocks, so that packets from multiple sessions are compressed together. It worked really well, it enabled me to offer reasonable fast content delivery to clients with servers on small leased lines in Africa. I found it a hard service to sell, it takes a lot of explaining ;-)