Load Balancing without Load Balancers
blog.cloudflare.com
Load Balancing without Load Balancers
1–10 of 33 posts
Re: Load Balancing without Load Balancers
#2> 2. Router: at the edge of each of our PoPs is a router. This router announces the paths packets take to CloudFlare's network from the rest of the Internet.
Umm, just curious, but I thought you used a set of redundant routers at each POP?
Or is the single router used highly redundant on its own?
Re: Load Balancing without Load Balancers
#3> 2. Router: at the edge of each of our PoPs is a router. This router announces the paths packets take to CloudFlare's network from the rest of the Internet. Umm, just curious, but I thought you used a set of redundant routers at each POP? Or is the single router used highly redundant on its own?
Typically a PoP could be defined as either an individual provider being brought into a facility or the location of a group of providers entering the facility (meet me room). In this instance I think they mean an individual provider (for example Level3) bringing in a feed into a data center. You would then get multiple feeds from distinct providers. Each feed would require a router for termination, and this router is typically using BGP. Since you keep multiple providers having redundant routers on each feed isn't always necessary. But there are cases where I've seen a single feed being served by redundant routers using something like VRRP. After that you would take feeds from the boarder router into our inner/core switch fabric to be distributed throughout your network. It just depends on the level of redundancy you want at each layer.
Re: Load Balancing without Load Balancers
#4> 2. Router: at the edge of each of our PoPs is a router. This router announces the paths packets take to CloudFlare's network from the rest of the Internet. Umm, just curious, but I thought you used a set of redundant routers at each POP? Or is the single router used highly redundant on its own?
Since this is all for a anycast system it might be better to have more POPs at more locations instead of trying to make a single POP bulletproof by doubling up on what could be the single largest capital expense in building a POP.
Re: Load Balancing without Load Balancers
#5> 2. Router: at the edge of each of our PoPs is a router. This router announces the paths packets take to CloudFlare's network from the rest of the Internet. Umm, just curious, but I thought you used a set of redundant routers at each POP? Or is the single router used highly redundant on its own?
Typically a PoP could be defined as either an individual provider being brought into a facility or the location of a group of providers entering the facility (meet me room). In this instance I think they mean an individual provider (for example Level3) bringing in a feed into a data center. You would then get multiple feeds from distinct providers. Each feed would require a router for termination, and this router is…
Thankyou. I remember VRRP being a bit of a mess from inception due to
the Cisco patent claims against the original IETF draft (too similar to
their patented HSRP). Are there still a lot of compatibility problems
across vendors? At home I used to use CARP (Common Address Redundancy
Protocol) on OpenBSD, but I haven't done it for awhile, and I know it's
improved a lot since then (i.e. pfsync(4), ifstated(8), ...).
Re: Load Balancing without Load Balancers
#6> 2. Router: at the edge of each of our PoPs is a router. This router announces the paths packets take to CloudFlare's network from the rest of the Internet. Umm, just curious, but I thought you used a set of redundant routers at each POP? Or is the single router used highly redundant on its own?
Since this is all for a anycast system it might be better to have more POPs at more locations instead of trying to make a single POP bulletproof by doubling up on what could be the single largest capital expense in building a POP.
This is generally our philosophy.
Re: Load Balancing without Load Balancers
#7One thing I've wondered is how homogenous your POPs are -- how big is the multiple (in router size/number and server count) between say San Jose POP and Sydney POP. I assume you'd want to scale based on expected traffic which considers each POP nearest, but that changes based on factors outside your control, and could change fairly fast. Plus DoS might come from areas which don't see a lot of regular traffic.
Re: Load Balancing without Load Balancers
#8Why not also use software load balancers? I don't see the advantage of going to so much effort to avoid software load balancing. It's neat that you guys got it to work, but I would think that a hybrid system would have more functionality and could better handle degraded performance situations.
Re: Load Balancing without Load Balancers
#9One thing I've wondered is how homogenous your POPs are -- how big is the multiple (in router size/number and server count) between say San Jose POP and Sydney POP. I assume you'd want to scale based on expected traffic which considers each POP nearest, but that changes based on factors outside your control, and could change fairly fast. Plus DoS might come from areas which don't see a lot of regular traffic.
Yes, our PoPs are different sizes depending on the load. San Jose and Los Angeles, for example, are larger than Seattle. We've designed the system to scale relatively linearly by adding additional servers. As a PoP gets more traffic we can handle it in two ways: 1) adding more equipment to the PoP; or 2) adding another PoP to offload a portion of the traffic.
Re: Load Balancing without Load Balancers
#10I'd be interested to see a blog post that details what "adjusting the way we handle [TCP] protocol negotiation itself [for Anycast]" entails.