NGINX open sources TCP load balancing • Hacker News Board

Re: NGINX open sources TCP load balancing

#101

Post by jospoortvliet » Tue, Apr 21, 2015, 9:12 AM UTC

dschiptsov wrote: post #89

This is, perhaps, a canonical example of how management's attempt to monetize an open source project will cause sub-optimal results both in code quality and profits. New features should be developed and tested in an open version, so the feedback, testing, patches and even unexpected new improvements from high skilled enthusiasts would be incorporated much more quickly than any closed team with QA (look at the Linux k…

I agree with you, but knowing a little how hard it is to stay smart about this as an open source company, I understand why it went wrong. The Open Source Way is surprisingly hard to understand and the pressure from Sales turns the focus on short term gains.

Re: NGINX open sources TCP load balancing

#102

Post by CrLf » Tue, Apr 21, 2015, 9:56 AM UTC

MichaelGG wrote: post #28

Maybe they'll bring in the health checks for HTTP balancing, next :).

It already supports it: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#...

Or you're referring to something else?

Re: NGINX open sources TCP load balancing

#103

Post by jfindley » Tue, Apr 21, 2015, 10:08 AM UTC

lobster_johnson wrote: post #38

toomuchtodo wrote: post #9

Can someone explain how this is superior to HAproxy?

HAProxy's primary feature is HTTP/HTTPS load balancing. This new feature competes only with HAProxy's TCP load balancing support. Note that Nginx already has a simple proxy built in that does very basic HTTP load balancing. HAProxy's is vastly superior to Nginx's in that it supports a sophisticated set of filters ("ACLs"), transformations (eg., header rewriting), queue behaviours (eg., queue limits, backup backends,…

> A big difference is that HAProxy's main balancing algorithm is "fair"

I don't think that's really true - haproxy has lots of load balancing modes, none of which are called "fair". It's also very unclear what you'd mean by "fair" in this context. Least connection? Maybe, but with a traffic pattern with large volumes of very short-lived requests, least connection actually won't be very fair at all, and will end up loading some servers more than others. Roundrobin isn't really "fair" either, if you have a mix of short and long requests.

Re: NGINX open sources TCP load balancing

#104

Post by jfindley » Tue, Apr 21, 2015, 10:19 AM UTC

jsmthrowaway wrote: post #67

jvehent wrote: post #41

Earlier quoted context omitted.

Surely you must have misread the docs, it's pretty damn powerful.

I ran 1.4 in production at a 8,000+ QPS social network, have been on a team who submitted patches to Tarreau that are now in HAproxy, and very intentionally put Openresty behind it for HTTP after months of tweaking a very fragile HAproxy configuration with several applications hanging off our property's domain name. I also architected and built a LBaaS product at a well-known hosting provider using HAproxy. I didn't…

I'm not entirely disagreeing with your post, but there's a few comments I'd like to make.

1) HAproxy does support SPDY, now via NPN/ALPN. And you don't need http mode for this.

2) HTTP performance (and performance in general) is now much better. Particularly on newer kernels (IIRC somewhere in the 3.12-3.15 range, when splice was fixed for small objects).

3. I'm not sure why you found it fragile. I've run HAProxy in HTTP mode at much higher QPS volumes than you were seeing - this did require a bunch of tuning, and I'm actually hoping to have time sometime in the next few weeks to write some articles on doing this. But it worked, and worked well.

What's worked well for me is the following:

           /-  [haproxy]
  [router] --  [haproxy]
           \-  [haproxy]

Using anycast routing to distribute the load over the haproxy servers, who run BIRD[1]. It's simple and pretty effective - although you want to chose your router hashing method carefully, and if you use persistance (stick tables) you have to use a recent version of haproxy, with support for peering.

It is difficult to make it work, though - just how difficult I only recently appreciated when trying to help a friend over IRC make haproxy scale to very high volumes. I'm hoping that I'll find the time to write the articles I mentioned above, which will hopefully be useful to others with this sort of problem to solve.

1: http://bird.network.cz/

Re: NGINX open sources TCP load balancing

#105

Post by IgorPartola » Tue, Apr 21, 2015, 12:16 PM UTC

dschiptsov wrote: post #89

This is, perhaps, a canonical example of how management's attempt to monetize an open source project will cause sub-optimal results both in code quality and profits. New features should be developed and tested in an open version, so the feedback, testing, patches and even unexpected new improvements from high skilled enthusiasts would be incorporated much more quickly than any closed team with QA (look at the Linux k…

Not that I don't want me some more nginx features, but how will this work exactly? Are you suggesting these features, once developed, become closed source? Honestly, I am not sure how nginx could be profitable long term. It is so good, that you don't need paid support or whatever the Plus version offers.

Re: NGINX open sources TCP load balancing

#106

Post by gizzlon » Tue, Apr 21, 2015, 1:17 PM UTC

IgorPartola wrote: post #105

dschiptsov wrote: post #89

This is, perhaps, a canonical example of how management's attempt to monetize an open source project will cause sub-optimal results both in code quality and profits. New features should be developed and tested in an open version, so the feedback, testing, patches and even unexpected new improvements from high skilled enthusiasts would be incorporated much more quickly than any closed team with QA (look at the Linux k…

Not that I don't want me some more nginx features, but how will this work exactly? Are you suggesting these features, once developed, become closed source? Honestly, I am not sure how nginx could be profitable long term. It is so good, that you don't need paid support or whatever the Plus version offers.

Guess it would work more like Fedora -> Redhat Enterprise Linux. You might be right about the profits, I have no idea..

The "open core" model is horrible IMO. It pits the open source version and the commercial version againts each other. What happens when someone would like to contribute features already planned for the commercial version.. ?

Re: NGINX open sources TCP load balancing

#107

Post by dschiptsov » Tue, Apr 21, 2015, 1:35 PM UTC

IgorPartola wrote: post #105

dschiptsov wrote: post #89

This is, perhaps, a canonical example of how management's attempt to monetize an open source project will cause sub-optimal results both in code quality and profits. New features should be developed and tested in an open version, so the feedback, testing, patches and even unexpected new improvements from high skilled enthusiasts would be incorporated much more quickly than any closed team with QA (look at the Linux k…

Not that I don't want me some more nginx features, but how will this work exactly? Are you suggesting these features, once developed, become closed source? Honestly, I am not sure how nginx could be profitable long term. It is so good, that you don't need paid support or whatever the Plus version offers.

I would say that open source code base cannot be profitable in principle (and it should be completely open, like the Linux kernel, with all the new "hot" features development) but the services, you as a developer provide, could.

Or let's say that one's knowledge and skills are profitable, not the code itself.

The code should be open, so it could grow on the same pace as everything else grows, according to the real demand for features, like early FreeBSD or Linux or even MySQL grew, or even nginx before 1.0.

Re: NGINX open sources TCP load balancing

#108

Post by dschiptsov » Tue, Apr 21, 2015, 1:40 PM UTC

gizzlon wrote: post #106

IgorPartola wrote: post #105

Earlier quoted context omitted.

Not that I don't want me some more nginx features, but how will this work exactly? Are you suggesting these features, once developed, become closed source? Honestly, I am not sure how nginx could be profitable long term. It is so good, that you don't need paid support or whatever the Plus version offers.

Guess it would work more like Fedora -> Redhat Enterprise Linux. You might be right about the profits, I have no idea.. The "open core" model is horrible IMO. It pits the open source version and the commercial version againts each other. What happens when someone would like to contribute features already planned for the commercial version.. ?

Redhat has really clever model, btw. Once, in times of RHEL3 and 4 they have tried to maintain zillion patches against vanilla kernel to be "Enterprise Linux", you know, so you could run Oracle cheap (a hot topic in that time). Then they have realized, that it is much smarter to give the patches to the mainstream, so everyone will benefit.

Fedora became a test-bed for a new technologies, to amortize the too rapid changes (systemd and other crap, you know), so they could provide stable and compatible RHEL versions for existing customers.

And Redhat is a service, not the code company.

Re: NGINX open sources TCP load balancing

#109

Post by IgorPartola » Tue, Apr 21, 2015, 1:56 PM UTC

dschiptsov wrote: post #107

IgorPartola wrote: post #105

Earlier quoted context omitted.

Not that I don't want me some more nginx features, but how will this work exactly? Are you suggesting these features, once developed, become closed source? Honestly, I am not sure how nginx could be profitable long term. It is so good, that you don't need paid support or whatever the Plus version offers.

I would say that open source code base cannot be profitable in principle (and it should be completely open, like the Linux kernel, with all the new "hot" features development) but the services, you as a developer provide, could. Or let's say that one's knowledge and skills are profitable, not the code itself. The code should be open, so it could grow on the same pace as everything else grows, according to the real de…

Well, I think an OSS piece of software can be profitable if you can run it as a service. For example, WordPress (wordpress.com sells subscriptions), or Sentry (getsentry.com). Sure you can run your own, but why not pay someone to take care of that headache?

However, I am looking specifically at nginx. The open version of it is too good and too complete, and by its nature it's not something you are going to run externally to your application. It's what Alan Cox (I think) described as "legacy software", meaning it's a fundamental part of your infrastructure that you expect to just sit there and work.

Re: NGINX open sources TCP load balancing

#110

Post by lobster_johnson » Tue, Apr 21, 2015, 2:16 PM UTC

jfindley wrote: post #103

lobster_johnson wrote: post #38

Earlier quoted context omitted.

HAProxy's primary feature is HTTP/HTTPS load balancing. This new feature competes only with HAProxy's TCP load balancing support. Note that Nginx already has a simple proxy built in that does very basic HTTP load balancing. HAProxy's is vastly superior to Nginx's in that it supports a sophisticated set of filters ("ACLs"), transformations (eg., header rewriting), queue behaviours (eg., queue limits, backup backends,…

> A big difference is that HAProxy's main balancing algorithm is "fair" I don't think that's really true - haproxy has lots of load balancing modes, none of which are called "fair". It's also very unclear what you'd mean by "fair" in this context. Least connection? Maybe, but with a traffic pattern with large volumes of very short-lived requests, least connection actually won't be very fair at all, and will end up lo…

Well, "fair" was in quotes for a reason. My only point was something that HAProxy has something that distributes based on state, as opposed to round robin or random distribution.