Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HAProxy 2.4

haproxy.com

Post reply on HN
Print view View original

61–70 of 70 posts

Re: HAProxy 2.4

#61

Post by gregmac »

smitty1e wrote: post #2

If I have a cloud provider, why might I use HAProxy over the provider's load balancing solution?

We use it at work for hosting an app designed pre-cloud (using one db per tenant) in AWS:

We deploy auto-scaling groups for each major app version (because customers can choose when to upgrade, so we can have anywhere from 1-3 versions live at a time). There's a database that has an entry for each customer system, with one or more domains, the app version and the db name (usually auto-generated but for historical reasons can be set manually). There's a UI to manage all this.

A script takes this data and builds haproxy config, creating backends for each version group and routing rules for domain to the proper group. This part could maybe be done with ALB now, but I am not certain of that.

We also automatically configure SSL for all domains: anything that doesn't have a static .pem file gets LetsEncrypt cert. Most of these are done via HTTP-01 because they're customer-owned domains that just CNANE to us. None of this is doable via AWS built-in stuff.

There's also a bunch of other hacks that haproxy does:

doesn't redirect to SSL for a couple specific (non sensitive) URLs+user agent that doesn't follow redirects;

returns a fake "success" page for a long-gone service called by an obsolete client some customers of (ex-)customers are still running, which effective causes a DDoS attack due to retries if we return 404 or 5xx;

Has some awareness of backend state and shows better error pages than just a generic 5xx depending on situation.

Haproxy instances are behind NLB, but otherwise there's a single (layer 7) hop to the app server.

The end result is you can configure a new system via our management UI, and so long as DNS is setup (using a wildcard subdomain we own, and/or customer's CNAME entry) within a few minutes it will be live (database deployed, app servers aware of connection/domain mapping, proxy configured with SSL).

Re: HAProxy 2.4

#62

Post by bullen »

nijave wrote: post #49
bullen wrote: post #33

Earlier quoted context omitted.

The only "real" reason you need a loadbalancer is that you want to save IP addresses. If all your machines have their own unique public address you can loadbalance with your own software by pointing people directly to different machines.

Even with scarce IPv4 you could currently do that just running backends on different ports It's common to do internal services that way but you can't control how or who or the internet might try to connect to your service so the load balancer offers a layer of protection so servers can't be picked off individually. They can also offer an abstraction/routing layer so you can do things like canaries and controlled roll…

I'm going to reply to all of you here:

You cannot use different ports from the client, it has to be 80 for HTTP, I meant; do you own loadbalancing from the client!

So f.ex. the browser would load parts of the site from another server via javascript or redirect to another dns with another IPv6 address.

To have a bottleneck between all your clients and all your servers is not good architecture.

Re: HAProxy 2.4

#63

Post by rogerdonut »

aidos wrote: post #27
rogerdonut wrote: post #6

Earlier quoted context omitted.

disclaimer: I work at HAProxy Technologies More control/flexibility (i.e. cloud provider usually gives you a limited number of "checkboxes" that they want you to be able to modify), generally better observability (HAProxy timing metrics [1] and session state at disconnection codes [2] are very helpful for debugging), and granular control over timeouts [3]. You also get builtin rate limiting [4], which from what I hav…

I last used HAProxy about a decade ago for fairly simple load balancing and caching - worked really well. Question about the suitability for a dynamic backend case. I have a system where clients are assigned to backends exclusively for the duration of a session. At the moment I have a pool of backends and each client is explicitly told which backend to use so they can route to it. Ideally I’d make this transparent so…

While I have not tested this specific use case, I would start by exploring map files [1] and/or stick tables [2]. Map files can be updated on-the-fly through the HAProxy Runtime API. With stick tables you can persist a session/client based on anything found within the request headers, which would include a users session id.

[1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/

[2] https://www.haproxy.com/blog/introduction-to-haproxy-stick-t...

Re: HAProxy 2.4

#64

Post by aidos »

rogerdonut wrote: post #63
aidos wrote: post #27

Earlier quoted context omitted.

I last used HAProxy about a decade ago for fairly simple load balancing and caching - worked really well. Question about the suitability for a dynamic backend case. I have a system where clients are assigned to backends exclusively for the duration of a session. At the moment I have a pool of backends and each client is explicitly told which backend to use so they can route to it. Ideally I’d make this transparent so…

While I have not tested this specific use case, I would start by exploring map files [1] and/or stick tables [2]. Map files can be updated on-the-fly through the HAProxy Runtime API. With stick tables you can persist a session/client based on anything found within the request headers, which would include a users session id. [1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/ [2] https://www.haproxy.com/blo…

Excellent, thanks for the info, will check it out.

Re: HAProxy 2.4

#65

Post by toast0 »

bullen wrote: post #62
nijave wrote: post #49

Earlier quoted context omitted.

Even with scarce IPv4 you could currently do that just running backends on different ports It's common to do internal services that way but you can't control how or who or the internet might try to connect to your service so the load balancer offers a layer of protection so servers can't be picked off individually. They can also offer an abstraction/routing layer so you can do things like canaries and controlled roll…

I'm going to reply to all of you here: You cannot use different ports from the client, it has to be 80 for HTTP, I meant; do you own loadbalancing from the client! So f.ex. the browser would load parts of the site from another server via javascript or redirect to another dns with another IPv6 address. To have a bottleneck between all your clients and all your servers is not good architecture.

> To have a bottleneck between all your clients and all your servers is not good architecture.

Maybe not, but everyone else is doing it. If your clients are web browsers, it's hard to convince them to do useful things like notice servers have changed; much easier to focus their attention on a handful of load balancers on static IPs and figure it out from there.

Re: HAProxy 2.4

#66

Post by vladvasiliu »

toast0 wrote: post #59
vladvasiliu wrote: post #55

Earlier quoted context omitted.

> If you're running HAProxy on AWS, be sure your AWS firewall rules are stateless, otherwise you may well run into the unpublished connection tracking limits. This. I got bit by this the other day, a bunch of servers were having very low CPU usage but new connections to them were failing. Turns out it was because they were hitting the conntrack limit. Changing the security group rules to allow all from all "fixed" it…

> Although after a quick test, I think there's another limit, also unpublished, as I still can't establish as many connections as I want on a smaller instance. Any chance this is an OS limit in your instance? Either a connection tracking firewall there, or FD/socket count limits? I don't have experience beyond the AWS connection tracking limits because I ran into them while testing something, but only found out how t…

I doubt that, because the same exact VM, after changing to a beefier instance type (c5.4xl, instead of the original t3.micro and later c5.large) accepted the connections without any problem.

If it's OS related, it may be some dynamic limit based on RAM size. But the limit was something around 100-200 connections, so nothing particularly crazy.

The way I found out about the limit was that downstream servers couldn't establish connections, and a manual test showed significant packet loss, while the servers were under 10% CPU and accepted connections locally. I opened a support ticket and the guy apparently saw that they were hitting the conntrack limit.

Once I replaced the security group with a blanket allow all as he suggested, connections could be established again. But two days later, when the number of connections increased further (while the security group was still allow all), I started seeing again connections not being established, although the servers were still not particularly loaded. As these are UDP (DNS) servers, I can't usefully quantify the number of different clients.

I set up a quick lab with two isolated EC2 instances and figured there must be another limit to the number of connections, although it seems somewhat higher – or maybe the conntrack limit as shown by ethtool doesn't update immediately. This happened yesterday, Saturday, while I was on call, so I didn't push the tests far enough to reach any clear-cut conclusion. I just doubled the servers and everything went back to normal.

However, the limit does seem to be related to the instance size and per interface. Meaning that adding another interface doubled the connections I could establish. Again, I didn't push the tests any further, though I intend to this sometime next week.

Re: HAProxy 2.4

#67

Post by NDizzle »

detaro wrote: post #53
NDizzle wrote: post #52

Earlier quoted context omitted.

Downvote me all you want but “architects” need to look at the solution landscape at least once in 15 years.

Plenty areas of software where HAProxy is not part of the relevant "solution landscape".

Systems architect?! I want a load balancer. What’s your top 5 excluding HAProxy?

Re: HAProxy 2.4

#68

Post by throwawaysea »

rogerdonut wrote: post #7

Author of the blog post here, happy to answer any questions about this release.

As someone who isn’t familiar with this space - is the use of HAProxy as a self hosted DDOS defense a “supported” (prioritized?) use case? I found old blog posts on this application of HAProxy but wasn’t sure if new development tries to improve those capabilities or if it is more focused on load balancing or other applications.

Re: HAProxy 2.4

#69

Post by nickramirez »

kzrdude wrote: post #37

Any blog post or release announcement that is expected to get wide dissemination should preferably have a blurb that explains what the project/product is , so that the post is accessible to more people. Maybe something like "The Reliable, High Performance TCP/HTTP Load Balancer".

kzrdude, great feedback. These big version announcements gather the attention of people who may not be familiar with HAProxy and so, I'd say they are a unique case where we should introduce the product for newcomers.

Re: HAProxy 2.4

#70

Post by AdrianB1 »

NDizzle wrote: post #67
detaro wrote: post #53

Earlier quoted context omitted.

Plenty areas of software where HAProxy is not part of the relevant "solution landscape".

Systems architect?! I want a load balancer. What’s your top 5 excluding HAProxy?

You don't need 5, especially when the load balancing is part of the network or infrastructure team that are a different group. Some companies have hardware load balancers, some use products like VMWare NSX or both, depending on the needs. When you are a Windows shop, HAProxy is not on the short list, hearing about it and actually knowing it are 2 different sides of the coin.
Post reply on HN