Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Networking on AWS (2018)

grahamlyons.com

Post reply on HN
Print view View original

71–80 of 104 posts

Re: Networking on AWS (2018)

#71

Post by jrockway »

iheartpotatoes wrote: post #61

Ugh! That's the most complex explanation of AWS I've ever seen. He just described a NETWORK, not AWS. AWS has renamed lots of things, but all the scary text configs that used to be the domain of wizened sysadmins have been replaced with very simple single-page-app GUI controls. LIke routers and gateways: those terms are largely gone from the AWS vocabulary. No need to get into subnets and route tables I think. The ma…

Having just gone through the process of using EKS, which requires a VPC, I think the article is quite applicable to anyone doing anything of average complexity. I found myself quite often wondering things like "do I have to run an Internet Gateway in every availability zone?" (no) and "do I attach my NAT Gateway to the public subnet or the private subnet in each AZ?" (public, and then add an entry for the gateway in the private subnet).

Amazon does not clearly document any of this. (I think if you read enough you'll eventually figure it out, but experimentation was the most straightforward procedure here).

As for just using a single EC2 instance and RDS... that is something you can do, but not everyone's workload is so simple that they can run it on one machine. And not everyone can afford do be down simply because one AZ is down. Hence, multi-AZ VPC setups.

Re: Networking on AWS (2018)

#73

Post by wslh »

I want to add a few notes useful for packet crafting. AWS, Google Cloud, and Azure don't work at layer 2 (Ethernet) as expected since they provide services at layer 3 and up.

For example, if you modify the MAC destination address it will not work in AWS. To be able to do that you should disable source/destination checks as is specified in [1].

The last time I checked you cannot do that in Google Cloud or Microsoft Azure.

When we experienced this issue, Reddit was the best resource for answers. I put the Reddit threads as they can help others working in projects requiring packet crafting:

https://www.reddit.com/r/sysadmin/comments/51xypj/vpc_amazon...

https://www.reddit.com/r/networking/comments/51y52n/aws_vpc_...

https://www.reddit.com/r/sysadmin/comments/533e14/google_com...

[1] https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Ins...

Re: Networking on AWS (2018)

#74

Post by CloudNetworking »

late2part wrote: post #68
bg24 wrote: post #30

Earlier quoted context omitted.

I do not and did not work for Microsoft, but love this very attitude.

Well, it’s pointless, really. They’ve been given feedback that blocking I Mp is harmfulmsince 2011 and it’s done no good.

Disclaimer: I have worked for Azure in the past -not anymore- and specifically in Networking.

Do you have evidence that they've been saying they block ICMP because of security reasons that I can forward to the right folks? I can help getting this feedback to them to correct that, because I can guarantee 100% that's not the reason why ICMP is not forwarded by the SLB (and engineering/PM would never say it's for security reasons).

Re: Networking on AWS (2018)

#75

Post by CloudNetworking »

llama052 wrote: post #26

Off topic, but as a network guy by heart I've always been fairly happy with how AWS implements the network side of things, especially in comparison to something like Azure. AWS you have the same basic concepts of a network, and the terminology aligns enough that you can make sense of it fairly quick if you're in the network realm. Azure however takes all of that 'network' stuff and turns it into this abstraction wher…

Disclaimer: I have worked for Azure in the past -not anymore- and specifically in networking.

There's a few things I agree (the UI, even though I like it more than the AWS UI, is not my all time favourite GUI application) and others I disagree, but I wanted to correct some things about your specific example as I'm sure you'll find the information useful:

> In Azure, I'd use expressroute, add the Expressroute circuit to a Subscription,

Anything you create is created inside a subscription and those are not separate steps.

> add a gateway for that, and then an additional gateway for each VPC equivalent,

That's accurate and by design. Each VNet needs their ExpressRoute gateway - that's unless you peer those VNets, then you can use the "hub vnet"'s gateway for all of them, e.g.: https://docs.microsoft.com/en-us/azure/architecture/referenc...

> create an authorization key for each 'VPC' equivalent and sync them,

Only if those VNets belong to other subscriptions, that's why you have to authorize them as the owner of the ExpressRoute. Depending on how your company has structured their access to Azure (they might have a subscription only for ER circuits, I've seen that a few times) you'll need to do this or not.

> and then define routing per gateway.

Not sure what you mean by "define routing". ExpressRoute uses BGP and learns routes from on-prem and from the VNet itself.

> Then when you go in to trace the network path ICMP is blocked.

ICMP is not forwarded by the Azure Load Balancer, but the load balancer is not in the path of traffic between on-premises and your VNets. You totally be able to traceroute to/from on-prem in an ExpressRoute scenario.

Also, as per the transit gateway I believe the equivalent might be the Virtual Want https://docs.microsoft.com/en-us/azure/virtual-wan/virtual-w... but I haven't really dug much into it yet. As far as I'm aware it doesn't support ExpressRoute at the moment.

All in all I believe your opinion is biased due to what you're used to (unconscious bias, plz don't get me wrong). I had the similar feeling when going to AWS, but after using it daily for some time you see the rationale behind the (different) product design decisions. We're humans, that's how our brain works! :)

Now on the other hand, I've got this daily AWS exposure for the last few months and it's very similar networking-wise. Heck, I even guessed existing features based on troubleshooting + my previous Azure knowledge (I'm looking at you "Src dst check" https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-en... - AKA "IP Forwarding" in Azure).

Re: Networking on AWS (2018)

#76

Post by CloudNetworking »

wslh wrote: post #73

I want to add a few notes useful for packet crafting. AWS, Google Cloud, and Azure don't work at layer 2 (Ethernet) as expected since they provide services at layer 3 and up. For example, if you modify the MAC destination address it will not work in AWS. To be able to do that you should disable source/destination checks as is specified in [1]. The last time I checked you cannot do that in Google Cloud or Microsoft Az…

And for those curious as to how/why this works this way, Azure has published papers on their virtual switch: https://www.microsoft.com/en-us/research/project/azure-virtu...

Re: Networking on AWS (2018)

#77

Post by CloudNetworking »

iheartpotatoes wrote: post #61

Ugh! That's the most complex explanation of AWS I've ever seen. He just described a NETWORK, not AWS. AWS has renamed lots of things, but all the scary text configs that used to be the domain of wizened sysadmins have been replaced with very simple single-page-app GUI controls. LIke routers and gateways: those terms are largely gone from the AWS vocabulary. No need to get into subnets and route tables I think. The ma…

The fact that you don't need to know it because you only work with very small customers doesn't mean other folks don't need it :)

Op's article is great IMHO.

Re: Networking on AWS (2018)

#78

Post by Hikikomori »

tlynchpin wrote: post #49
llama052 wrote: post #26

Off topic, but as a network guy by heart I've always been fairly happy with how AWS implements the network side of things, especially in comparison to something like Azure. AWS you have the same basic concepts of a network, and the terminology aligns enough that you can make sense of it fairly quick if you're in the network realm. Azure however takes all of that 'network' stuff and turns it into this abstraction wher…

Transit Gateway was announced at re:invent 2018. That's a few months ago. DirectConnect Gateway and Transit VPC were announced the year before. I would guess about nobody is currently in production with Transit Gateway.

Currently testing it, can do lots of fun stuff with the routing domains. Will roll it out once they get Direct Connect support in Q1.

Re: Networking on AWS (2018)

#79

Post by curiouserrr »

tlynchpin wrote: post #49
llama052 wrote: post #26

Off topic, but as a network guy by heart I've always been fairly happy with how AWS implements the network side of things, especially in comparison to something like Azure. AWS you have the same basic concepts of a network, and the terminology aligns enough that you can make sense of it fairly quick if you're in the network realm. Azure however takes all of that 'network' stuff and turns it into this abstraction wher…

Transit Gateway was announced at re:invent 2018. That's a few months ago. DirectConnect Gateway and Transit VPC were announced the year before. I would guess about nobody is currently in production with Transit Gateway.

Except you can’t yet do cross-account transit, which makes their Transit VPC offering pretty much useless. The whole point of transit is so you can talk to other VPCs across accounts.

Re: Networking on AWS (2018)

#80

Post by devonkim »

The part that irks me is that if you’re doing any VPC design that’s going to even potentially include peering you need to carefully understand the limitations first. This means that within the same region you can refer to security groups in rules as if they’re in the same VPC, but you can’t do that if you’re peering across regions. Then add in some DNS restrictions (like not being able to directly resolve a peer VPC’s entries, somewhat solvable by use of VPC private zones to serve as DNS across regions) and it can be real awkward. Then there’s overlapping VPC CIDR issues (VPC Transit Gateways can only sorta help this)

The primary caveats beyond basic networks that impact designs is that multicasting is not enabled by the network layers but at the network interface (ENI) layer and you need to carefully look at how security groups really work (they’re attached to an ENI fundamentally, which is how you can route between networks with a single instance as long as it’s within the same AZ)

All of this I’ve found was completely disregarded / unknown by almost every company outside the F500 or high end tech start-ups when they first started with AWS and I’ve spent a lot of my career having to migrate production environments between VPCs so that we can get enough room to grow adequately. Making subnets as small as possible is not what you should be doing in AWS, folks. In fact, making them real small means you spent a fair bit of effort which means you decided to put in a lot of effort without stopping to read the documentation in earnest for a couple hours. And using a default VPC CIDR repeatedly from the console is a pretty grand way to make sure you can never let two VPCs communicate with each other via anything other than a third intermediate VPC that you’ll have to migrate to eventually.

Some of the overly-cautious networking approaches I’ve seen include making a VPC for every single application / service, using a NACL for every application (multiplied by every AZ used to isolate each subnet and cutting off cross-AZ routing thereby, of course), creating your own NAT instance that doesn’t do anything better than a NAT gateway, NAT gateways in every AZ (for a whole $1 of traffic / mo each). The story of problems in AWS infrastructure is the same - trying to plan too far ahead for the wrong things and not realizing the limitations of the right things that are not flexible anymore. This is much more common when companies hire traditionally experienced network engineers that have just a little too much confidence.

Post reply on HN