Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Billion User Load Balancer [video]

usenix.org

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Building a Billion User Load Balancer [video]

#41

Post by SEJeff »

noponpop wrote: post #16

Interesting that Linux kernel performance (ipvs) is acceptable at l4 vs something like dpdk. I guess you just overcome the limitation by increasing the number of l4 instances load balanced by ecmp. Fun to see DSR in use. Also interesting to see that all the inherent problems with geolocation via gslb (DNS client IP is not the same as the real client IP) don't wind up being a big problem apparently. This seems to be a…

Out of kernel offloads (dpdk, solarflare's openonload, and mellanox's VMA) are good for two primary use cases:

* Reducing context switches at exceptionally high packet rates * Massively reducing latency with tricks like busy polling (which the kernel's native stack is gaining)

LVS is pretty much the undisputed king for serious business load balancing. I've heard (anecdotally) that Uber uses gorb[1] and google has released seesaw, which are both fancy wrappers ontop of LVS for load balancing.

Source: Almost 10 years optimizing Linux and hardware for low latency in a trading firm.

[1] https://github.com/kobolog/gorb

[2] https://github.com/google/seesaw

Re: Building a Billion User Load Balancer [video]

#42

Post by bogomipz »

79d697i6fdif wrote: post #23
noponpop wrote: post #16

Interesting that Linux kernel performance (ipvs) is acceptable at l4 vs something like dpdk. I guess you just overcome the limitation by increasing the number of l4 instances load balanced by ecmp. Fun to see DSR in use. Also interesting to see that all the inherent problems with geolocation via gslb (DNS client IP is not the same as the real client IP) don't wind up being a big problem apparently. This seems to be a…

It's mostly because the point of DPDK and similar is to go around a lot of the processing in kernel, and IPVS does exactly this. I'm surprised IPVS isn't more popular, it's built into the kernel and extremely fast. HTTP proxy type load balancers are slugs in comparison Scaling app servers to nearly unlimited size is easy to explain but really hard in practice. It basically amounts to this: 1) Balance requests using D…

>"'Im surprised IPVS isn't more popular, it's built into the kernel and extremely fast."

I feel it actually is popular at places that do 10's of Gigs of traffic and up, usually in combination with a routing daemon - Bird, Quagga etc. I have worked in couple of shops now that utilized a similar architecture. I also read recently about a Google LB that leveraged IPVS and now this of course.

Re: Building a Billion User Load Balancer [video]

#43

Post by bogomipz »

woodcut wrote: post #38
79d697i6fdif wrote: post #23

Earlier quoted context omitted.

It's mostly because the point of DPDK and similar is to go around a lot of the processing in kernel, and IPVS does exactly this. I'm surprised IPVS isn't more popular, it's built into the kernel and extremely fast. HTTP proxy type load balancers are slugs in comparison Scaling app servers to nearly unlimited size is easy to explain but really hard in practice. It basically amounts to this: 1) Balance requests using D…

What if you're not dealing with millions of connections but instead only a few thousand from whitelisted IP's and you need to optimise for high availability & latency? Could it be done with just anycast -> IPVS layer -> app servers ?

If its stateless traffic then yes.

The ECMP/Anycast just gets you beyond the limit of an single pair of IPVS boxes which are are kept in sync with keepalived/vrrp for HA.

But a pair of boxes with ipvs + keepalived + iptables should be be able to handle a few thousand connections no problems. Your concern would then likely be the bandwidth going through the box. But if your client pull rather than push using direct server return should be able to get you past the bandwidth limitations of a single box.

Re: Building a Billion User Load Balancer [video]

#44

Post by bogomipz »

SEJeff wrote: post #41
noponpop wrote: post #16

Interesting that Linux kernel performance (ipvs) is acceptable at l4 vs something like dpdk. I guess you just overcome the limitation by increasing the number of l4 instances load balanced by ecmp. Fun to see DSR in use. Also interesting to see that all the inherent problems with geolocation via gslb (DNS client IP is not the same as the real client IP) don't wind up being a big problem apparently. This seems to be a…

Out of kernel offloads (dpdk, solarflare's openonload, and mellanox's VMA) are good for two primary use cases: * Reducing context switches at exceptionally high packet rates * Massively reducing latency with tricks like busy polling (which the kernel's native stack is gaining) LVS is pretty much the undisputed king for serious business load balancing. I've heard (anecdotally) that Uber uses gorb[1] and google has rel…

Mellanox and Solarflare certainly have carved out a nice market for themselves. They are not cheap though which is I guess why they are mostly found in trading shops since latency likely equates to money being left on the table.

Re: Building a Billion User Load Balancer [video]

#45

Post by luizbafilho »

For those who are interested in replicate the very same architecture in your environment, I am working on a similar solution. Please check it out

https://github.com/luizbafilho/fusis

It is a control plane for IPVS and adds distribution, fault tolerance, self-configuration and a nice JSON API to it.

It is almost done, but it needs documentation on how to use it.

Moreover, for those who like numbers, I did some benchmarks using a 16 cores machine with two bounded 10Gbit interface

Scenarios:

1 request per connection with 1 byte: 115k connections/s

20 requests per connection with 1 byte: 670k requests/s

20 requests per connection with 1 megabyte: 14Gbps

It scenario tests one specific aspect of the load balancer.

Re: Building a Billion User Load Balancer [video]

#46

Post by [deleted] »

patrickshuff wrote: post #31
ju-st wrote: post #14

The "cartographer" doesn't use end user related information to select the best PoP? Or does "Sonar" measure the latency/throughput by looking at existing connections to users? Ok I watched the presentation now, Sonar apparently does that indeed.

Sonar is the system we use to measure latency from the client devices to all of our PoPs. Cartographer is the system that consumes all of these sonar measurements and uses several other real time data sources (BGP routes, Link capacity, PoP health, PoP capacity, etc) and continually generates a GLB map for the most optimal targetting of requests to our PoPs.

[deleted]

Re: Building a Billion User Load Balancer [video]

#47

Post by bogomipz »

patrickshuff wrote: post #31
ju-st wrote: post #14

The "cartographer" doesn't use end user related information to select the best PoP? Or does "Sonar" measure the latency/throughput by looking at existing connections to users? Ok I watched the presentation now, Sonar apparently does that indeed.

Sonar is the system we use to measure latency from the client devices to all of our PoPs. Cartographer is the system that consumes all of these sonar measurements and uses several other real time data sources (BGP routes, Link capacity, PoP health, PoP capacity, etc) and continually generates a GLB map for the most optimal targetting of requests to our PoPs.

I am curious does Cartographer also adjust iBGP preferences for balancing amongst transit providers for egress traffic?

Re: Building a Billion User Load Balancer [video]

#48

Post by nrjdhsbsid »

woodcut wrote: post #38
79d697i6fdif wrote: post #23

Earlier quoted context omitted.

It's mostly because the point of DPDK and similar is to go around a lot of the processing in kernel, and IPVS does exactly this. I'm surprised IPVS isn't more popular, it's built into the kernel and extremely fast. HTTP proxy type load balancers are slugs in comparison Scaling app servers to nearly unlimited size is easy to explain but really hard in practice. It basically amounts to this: 1) Balance requests using D…

What if you're not dealing with millions of connections but instead only a few thousand from whitelisted IP's and you need to optimise for high availability & latency? Could it be done with just anycast -> IPVS layer -> app servers ?

Yeah it works pretty much the same. If your clients aren't geographically dispersed replace anycast with DNS round robin or use both like most huge sites do.

Also there's three layers :) dns->ipvs->httpproxy->app servers.

You could ditch the HTTP proxy layer if your app servers are extremely fast like netty/go/grizzly.

Re: Building a Billion User Load Balancer [video]

#49

Post by skuhn »

bogomipz wrote: post #44
SEJeff wrote: post #41

Earlier quoted context omitted.

Out of kernel offloads (dpdk, solarflare's openonload, and mellanox's VMA) are good for two primary use cases: * Reducing context switches at exceptionally high packet rates * Massively reducing latency with tricks like busy polling (which the kernel's native stack is gaining) LVS is pretty much the undisputed king for serious business load balancing. I've heard (anecdotally) that Uber uses gorb[1] and google has rel…

Mellanox and Solarflare certainly have carved out a nice market for themselves. They are not cheap though which is I guess why they are mostly found in trading shops since latency likely equates to money being left on the table.

High frequency trading is Solarflare's original market (and Mellanox has their InfiniBand market), but both of them are becoming more and more common in the commodity server market as well. Particularly since Intel dropped the ball on 40G (and REALLY dropped the ball on 25/100G), and Broadcom is out of the adapter market, there is a void that other vendors are filling.

When you're spending $30,000 on a server, it doesn't really matter if you spend $1200 on a network card. Those CPU cycles and storage bytes have to go somewhere to make money.

Re: Building a Billion User Load Balancer [video]

#50

Post by bogomipz »

skuhn wrote: post #49
bogomipz wrote: post #44

Earlier quoted context omitted.

Mellanox and Solarflare certainly have carved out a nice market for themselves. They are not cheap though which is I guess why they are mostly found in trading shops since latency likely equates to money being left on the table.

High frequency trading is Solarflare's original market (and Mellanox has their InfiniBand market), but both of them are becoming more and more common in the commodity server market as well. Particularly since Intel dropped the ball on 40G (and REALLY dropped the ball on 25/100G), and Broadcom is out of the adapter market, there is a void that other vendors are filling. When you're spending $30,000 on a server, it doe…

I totally agree. It sounds like their price points @ $1,200 have come down as well.

Did Intel just never get 25/100G card to market?

Post reply on HN