Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Network Load Balancer

aws.amazon.com

Post reply on HN
Print view View original

91–100 of 122 posts

Re: AWS Network Load Balancer

#91

Post by la6470 »

GeneticGenesis wrote: post #89
newhere420 wrote: post #79

Earlier quoted context omitted.

If they won't warn us, could you please warn us? - Fellow ALB user.

Sure. Whenever a "config change" (Note: this includes adding or removing targets to a target group, EG Autoscaling) happens on an ALB, the ALB drops all active connections, and re-establishes them at once, at high load, this obviously causes significant load spikes on any underlying service. You can see this happening by looking at the "Active Connection Count" graphs from your ALB, and adding or removing an instance…

And how often you do confit changes?

Re: AWS Network Load Balancer

#92

Post by acejam »

Thaxll wrote: post #27
colmmacc wrote: post #17

Earlier quoted context omitted.

ELB's Application Load Balancer (ALB) supports HTTP/2 termination. More in the original launch announcement: https://aws.amazon.com/blogs/aws/new-aws-application-load-ba... and https://aws.amazon.com/elasticloadbalancing/details/#details

http2 termination is not very useful, most people want ELB -> backend instances.

Yes it is. For the client, multiplexing multiple requests across a single connection and only needing to negotiate TLS once can have huge benefits, depending on the page/site. You are after all building your app/site for your end users, right?

Re: AWS Network Load Balancer

#93

Post by jdc0589 »

stelund wrote: post #71
colmmacc wrote: post #2

If you're curious to see NLB in action, here's a live demo: http://nlb-34dc3b430638dc3e.elb.us-west-2.amazonaws.com/ , it took about 5 minutes in the console to set it up and no changes on the targets/backends. Massive disclaimer: I work on NLB.

There is no Security Group for NLB, how is that reasoned?

that threw me for a bit of a loop as well. This means that responsibility for doing ACL whitelisting at the edge is now moved from the actual edge, to the security groups on the actual servers responding to request, right?

That's do-able and all, but I kind of didn't hate the old paradigm of having an extra layer there.

Re: AWS Network Load Balancer

#94

Post by jacobr1 »

paulddraper wrote: post #41

Static IP, source IP, and zonality are game changing. Unfortunately, it lacks a very significant existing feature of ELB: SSL/TLS termination. It's very convient to manage the certs in AWS without having to deploy them to dedicated EC2 instances.

This is what their ALB service if for.

Re: AWS Network Load Balancer

#95

Post by colmmacc »

jdc0589 wrote: post #93
stelund wrote: post #71

Earlier quoted context omitted.

There is no Security Group for NLB, how is that reasoned?

that threw me for a bit of a loop as well. This means that responsibility for doing ACL whitelisting at the edge is now moved from the actual edge, to the security groups on the actual servers responding to request, right? That's do-able and all, but I kind of didn't hate the old paradigm of having an extra layer there.

One way to think of NLB is that it's an Elastic IP address that happens to go to multiple instances or containers, instead of just one. Everything else stays the same.

Re: AWS Network Load Balancer

#97

Post by colmmacc »

deafcalculus wrote: post #87
krallin wrote: post #81

Earlier quoted context omitted.

AWS provides you with a number of DNS records for each NLB: - One record per zone (which maps to the EIP for that zone) - A top-level record that includes all active zones (these are all zones you have registered targets in, IIRC) The latter record is health checked, so if an AZ goes down, it'll stop advertising it automatically (there will be latency of course, so you'll have some clients connecting to a dead IP, bu…

Won't DNS failover be painfully slow? Some clients ignore small TTL values. I've seen DNS updates taking several hours to propagate. I thought one of the advantages of multiple zones is that zonal failover can happen with "zero" downtime (this seems to be the case with Amazon RDS).

The default answer includes multiple A records, so if clients can't reach one of the IPs, they try another. There's no need for anything to propagate for that to kick in, it's just ordinary client retry behavior.

We do also withdraw an IP from DNS if it fails; when we measure it, we see that over 99% of clients and resolvers do honor TTLs and the change is effected very quickly. We've been using this same process for www.amazon.com for a long time.

Contrast to an alternative like BGP anycast, where it can take minutes for an update to propagate as BGP peers share it with each other in sequence.

Re: AWS Network Load Balancer

#99

Post by mooreds »

deafcalculus wrote: post #87
krallin wrote: post #81

Earlier quoted context omitted.

AWS provides you with a number of DNS records for each NLB: - One record per zone (which maps to the EIP for that zone) - A top-level record that includes all active zones (these are all zones you have registered targets in, IIRC) The latter record is health checked, so if an AZ goes down, it'll stop advertising it automatically (there will be latency of course, so you'll have some clients connecting to a dead IP, bu…

Won't DNS failover be painfully slow? Some clients ignore small TTL values. I've seen DNS updates taking several hours to propagate. I thought one of the advantages of multiple zones is that zonal failover can happen with "zero" downtime (this seems to be the case with Amazon RDS).

RDS failover still uses DNS and you still need to be aware of client TTLs:

"Because the underlying IP address of a DB instance can change after a failover, caching the DNS data for an extended time can lead to connection failures if your application tries to connect to an IP address that no longer is in service."

http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_B...

Re: AWS Network Load Balancer

#100

Post by colmmacc »

GeneticGenesis wrote: post #89
newhere420 wrote: post #79

Earlier quoted context omitted.

If they won't warn us, could you please warn us? - Fellow ALB user.

Sure. Whenever a "config change" (Note: this includes adding or removing targets to a target group, EG Autoscaling) happens on an ALB, the ALB drops all active connections, and re-establishes them at once, at high load, this obviously causes significant load spikes on any underlying service. You can see this happening by looking at the "Active Connection Count" graphs from your ALB, and adding or removing an instance…

Wow that sounds awful - but thankfully this isn't typical. I'm going to go digging for a case-id/issue and see what's going on myself (please e-mail the case if you have one). Re-configurations are routine and graceful.

From your description it may be that you have long lived connections that build up over time, at a rate that targets can easily handle, but that the re-connect spikes associated with a target failure/withdrawal are too intense. This is a challenge I've seen with web sockets: imagine building up 100,000 mostly-idle web sockets slowly over time, even a modest pair of backends can handle this. But then a backend fails, and 50,000 connections come storming in at once!

Another scenario is adding an "idle" target to a busy workload, but it not being able to handle the increased rate of new connections it will get. Software that relies on caching (including things like internal object caches) often can handle a slow ramp-up, but not a sudden rush.

We're currently experimenting with algorithms that allow customers to more slowly ramp-up the incoming rate of connections in these kinds of scenarios.

Anyway, those are guesses, so I may be wrong about your case, but hopefully the information is still useful to others reading.

Post reply on HN