Live data from Hacker News

Load Balancing without Load Balancers

blog.cloudflare.com

31–33 of 33 posts

Re: Load Balancing without Load Balancers

#31
post #13

One thing I always wondered is how session persistence is maintained for things like webapps when trying to host services through anycast. Only idea I can dream up is cookie based and involves each site having internal proxy mappings to each pop in the event a different pop becomes the favored route (paths change all the time, and routers on the internet dont care that you had an established connection).

CloudFlare doesn't do sessions, they're just* a proxy cache to their customers' origins. * A nice proxy cache; and they do have some features to muck about with the content on the way through if you desire, but I don't see any application aware routing options.

What you're asking about isn't really an anycast problem either, you can have the same situation with any load balancing situation; if you need the client to come back to the same server, you need the client to bring you back something that says which server to route to, for example a cookie or a hostname, and you need something in your stack that handles that and routes it (could be DNS entries, a hardware or software load balancer, application logic, etc). Avoiding sessions is better, of course, if you can (or if you can have the client keep the state information; possibly encrypted and signed).

Re: Load Balancing without Load Balancers

#32
post #8

Why not also use software load balancers? I don't see the advantage of going to so much effort to avoid software load balancing. It's neat that you guys got it to work, but I would think that a hybrid system would have more functionality and could better handle degraded performance situations.

Exactly, as it's pretty much standard with other CDN's, and has been for a very long time. Without any application level intelligence, you would end up with duplicate copies of content, as every node in every city ends up caching every bit of content. You're then limited to the amount of cache each node has, resulting in mush less available cache overall, more content expiring sooner, and even more requests back to the origin server. Although, if you use a tiered approach with some intermediate nodes between the end cachers and the origin server, you could mitigate this somewhat but it would still be quite suboptimal. The more standard method is to use a 2 layer approach, with the front end first layer intelligently hashing the full URL across the pool of the back end second layer. The trickier part then is if a single object requires more than one back end node, you need to be hash the same content to more than one node. This could be done by just having multiple pools of server on the back end, if the scale requires it.

Re: Load Balancing without Load Balancers

#33
post #30
post #12

If a particular server starts to become overloaded, and it appears there is sufficient capacity elsewhere, then just some of the BGP routes can be withdrawn to take some traffic away from the overloaded server I'd be interested to hear about the mechanism for determining if there is sufficient capacity elsewhere, and how you avoid a cascading failure.

I'm interested in finding out what happens to tcp sessions that were established when routes are withdrawn.

This is the only thing I've found on this:

http://news.ycombinator.com/item?id=2484047

Post reply on HN