Live data from Hacker News

Networking on AWS (2018)

grahamlyons.com

31–40 of 104 posts

Re: Networking on AWS (2018)

#31
post #29

Earlier quoted context omitted.

A VPC with a public subnet that's locked down largely via security groups is probably fine for a project that can't justify a $35/month spend. edit: Apparently not. See below, my mistake.

It's counter intuitive but attaching a VPC Lambda to a public VPC subnet will not give it access to the internet. See: https://docs.aws.amazon.com/lambda/latest/dg/vpc.html#vpc-in...

Eww. TIL, thanks.

Re: Networking on AWS (2018)

#32
As long as IPv6 is a second-class citizen, things are going to continue to be painful in AWS.

I did the whole "here is your /56, now segment it yourself" thing. Its crude. It should not be neccessary, if V6 was central to the model, you'd be assigned /64 from your covering prefix automatically, as you deploy regional nodes.

Re: Networking on AWS (2018)

#33

There's quite a few things you've missed that are significant and should have been included, maybe one for part two: * Network ACLs, which describe the ruleset (consider it like a stateless firewall) for subnets and their respective routes. Whilst they are optional, having a default set it straightens out a lot of duplication that may end up in Security Groups (which are more stateful in nature). * Elastic (public) I…

Network ACLs [...] Whilst they are optional, having a default set it straightens out a lot of duplication that may end up in Security Groups (which are more stateful in nature).

I inherited an infrastructure that had NetACLs and security groups with duplicate entrypoints and policies, years of accumulated cruft because it was poorly designed and the documentation was even worse (read: nonexistent), security groups all the way down. That one threw me through a hard and annoying mental loop for a couple of hours until picking through with the finest tooth comb revealed what was going on.

The fun part is going to be rebuilding our routing in a new VPC such that it doesn't make the next guy want to put his head in a black hole.

I'd be lying if I said it wasn't a fun challenge in a sordid kind of way, though.

Re: Networking on AWS (2018)

#34
post #22
post #14

AWS security groups and ACLs are the most worthless things. you cant treat them like a real firewall. you end up just allowing anything outbound or inbound. they dont let you be detailed enough

Why are you just allowing anything outbound or inbound? You can specify Allow/Deny on any combination of source subnet, dest subnet, source port, dest port for starters. That gets you a pretty comprehensive ability to lock down a VPC on its own.

Say I want to allow outbound http/https to 10 different IPs. I can't do that in 1 rule like a traditional firewall.

Re: Networking on AWS (2018)

#35

Earlier quoted context omitted.

Having to reference security groups by id instead of name in cloudformation stacks, terraform, and a variety of other places is one of the most infuriating things. Makes everything much more difficult to configure and maintain because the IDs are so opaque and unique. Somebody has to do the grunt work of looking them and and copy/pasting them or writing scripts to propagate configuration forward. What a waste of time…

I agree, it's frustrating. However, considering you can share security groups, and even share VPC as a resource to different AWS accounts, and that "names" are non-unique, how would you solve the problem of allowing people to select a non-unique resource and know which one they actually mean. It's a pain, sure, but there's no better solution... explicitness creates certainty in this case..

You would use the arns. Those are unique but predictable.

Re: Networking on AWS (2018)

#36

NAT gateways are one of the things that blindsided me on the whole "serverless" idea for hobby projects. To have a Lambda function with access to the outside world and your private network resources your $0.01/month function becomes a $35/month+ expense if you don't want to manage your own t2 NAT instance (and required patches, upgrades, scaling, monitoring, etc). See https://forums.aws.amazon.com/thread.jspa?threadI…

Have you tried cloudflare workers? The networking gets taken care of for you, plus they are obscenely fast as they run on the edge closest to the client and use v8 isolates to drop 95th percentile latency from cold starts from ~1.5 seconds to about 300ms.

Hmm, cold starts for Workers depends on your script size, but should be around 10ms, never 300ms. Are you seeing 300ms? Is that actually Workers cold start time, or cold start for a larger application (e.g. that might include things like HTTP cache warm-up)?

(I'm the tech lead for Workers.)

Re: Networking on AWS (2018)

#38
Something that's missing from this (otherwise great!) guide, that has puzzled me for a while - what's the point? What does this configuration actually gain you/AWS? My best guess is that private subnets are for DDOS protection, but that seems like something that would be better handled by throttling. Given the amount of complaints I've heard about how difficult VPC/Subnet setup is, why bother with it at all? Staving off IP address exhaustion?

Or, to ask it another way - what would be the downside of all your resources being in 1 single-Subnet VPC, spread evenly across AZs?

Re: Networking on AWS (2018)

#39
post #36

Earlier quoted context omitted.

Have you tried cloudflare workers? The networking gets taken care of for you, plus they are obscenely fast as they run on the edge closest to the client and use v8 isolates to drop 95th percentile latency from cold starts from ~1.5 seconds to about 300ms.

Hmm, cold starts for Workers depends on your script size, but should be around 10ms, never 300ms. Are you seeing 300ms? Is that actually Workers cold start time, or cold start for a larger application (e.g. that might include things like HTTP cache warm-up)? (I'm the tech lead for Workers.)

I don’t think I explained it very well, sorry. I meant 300ms from an external load testing tool called wrk with 4 threads hitting it with 1,000 concurrent connections, including network round trip. This was only visible for the first run, after that I was seeing 99th percentile at ~90ms for the round trip. The worker I tested with is server rendering a react app. I’m away from the computer but I think the average speeds were ~20ms round trip. It’s crazy fast. I’m thrilled with it!

Re: Networking on AWS (2018)

#40
post #33

There's quite a few things you've missed that are significant and should have been included, maybe one for part two: * Network ACLs, which describe the ruleset (consider it like a stateless firewall) for subnets and their respective routes. Whilst they are optional, having a default set it straightens out a lot of duplication that may end up in Security Groups (which are more stateful in nature). * Elastic (public) I…

Network ACLs [...] Whilst they are optional, having a default set it straightens out a lot of duplication that may end up in Security Groups (which are more stateful in nature). I inherited an infrastructure that had NetACLs and security groups with duplicate entrypoints and policies, years of accumulated cruft because it was poorly designed and the documentation was even worse (read: nonexistent), security groups al…

I guess it’s a matter of preference, but I strongly prefer security groups over ACLs, which I don’t use at all. Even if only from a compliance perspective, a security group is equivalent to a host firewall (which personally helps me with PCI - no need for iptables and windows firewall). Whereas an ACL is a bit harder to make that case with. I also find them easier to audit.
Post reply on HN