Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Peerd: an AWS VPC peering connection management tool

github.com

Post reply on HN
Print view View original

21–28 of 28 posts

Re: Peerd: an AWS VPC peering connection management tool

#21

Post by c0restraint »

KenanSulayman wrote: post #13
kesor wrote: post #11

Earlier quoted context omitted.

Can you explain the "Patent Pending" bit? Why would you need to create a patent for a short Apache 2.0 licensed API calling Python script?

In addition, the apache license grants patent rights (§3 Grant of Patent License). Since it seems to be a research project, I assume they're just very happy to be able to put their name on it with Atlassian funding the patent :-)

Ya this isn’t something that should be patentable. Tons of prior art in conference talks on this. Cool tool though, glad it’s released!

Re: Peerd: an AWS VPC peering connection management tool

#22

Post by therockspush »

code4tee wrote: post #14

Cool tool and the problem it’s trying to solve is a real one. However on the complex mesh of connections across accounts isn’t the new way to do this now not the Transit Gateway? A tool that could manage complex TGW setups across many accounts would also be cool. Terraform can do that but it’s not the most elegant solution.

Terraform gets unruly at scale, and makes it hard to pass it off to other teams or take a vacation if you're responsible when it breaks.

Aviatrix popped up on my radar recently trying to solve the operations side of this. Handles cross account TGW attachments and simplifies the routing.

Re: Peerd: an AWS VPC peering connection management tool

#23

Post by windexh8er »

c0restraint wrote: post #21
KenanSulayman wrote: post #13

Earlier quoted context omitted.

In addition, the apache license grants patent rights (§3 Grant of Patent License). Since it seems to be a research project, I assume they're just very happy to be able to put their name on it with Atlassian funding the patent :-)

Ya this isn’t something that should be patentable. Tons of prior art in conference talks on this. Cool tool though, glad it’s released!

And there are commercial products that do this and more that have been around for years [0].

[0] https://www.aviatrix.com

Re: Peerd: an AWS VPC peering connection management tool

#25

Post by jcrites »

gregwebs wrote: post #20
jcrites wrote: post #17

Earlier quoted context omitted.

In addition to Transit Gateway, there's also AWS PrivateLink [1], which is my preferred approach for scenarios where it works. As an alternative to peering entire networks together, PrivateLink allows you to establish minimal connectivity between specific clients and services that need to communicate. When a service supports PrivateLink, any authorized client can create a private endpoint for that service in their VP…

How do you deal with the connections being to only one availability zone?

The connection can be present in multiple availability zones, though the service does need to decide which ones it will support at setup time. One simple approach is for the service to accept connections in many or all AZs, allowing the client to decide which ones are relevant to it. From the endpoint service documentation [1]:

> For low latency and fault tolerance, we recommend using a Network Load Balancer with targets in every Availability Zone of the AWS Region. To help achieve high availability (...) you can enable cross-zone load balancing. Cross-zone load balancing enables the load balancer to distribute traffic across the registered targets in all enabled Availability Zones. For more information, see Cross-Zone Load Balancing in the User Guide for Network Load Balancers.

When establishing a connection, the client can specify multiple subnets in which it would like to create an endpoint [2], and those subnets can be in any AZs supported by the service, if I recall correctly. The client can thus establish a connection that spans multiple AZ by requesting endpoints in subnets in those AZs. In most scenarios it's fine if client/service traffic can cross between AZs; I've only found it necessary to partition traffic by AZ in specialized applications (usually applications that are themselves trying to provide a zonal availability model, or that follow cell-based architecture at the zone level -- most business services don't fall into these categories, though low-level infrastructure services sometimes do).

[1] https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-se...

[2] https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_C...

Re: Peerd: an AWS VPC peering connection management tool

#26

Post by Terretta »

c0restraint wrote: post #21
KenanSulayman wrote: post #13

Earlier quoted context omitted.

In addition, the apache license grants patent rights (§3 Grant of Patent License). Since it seems to be a research project, I assume they're just very happy to be able to put their name on it with Atlassian funding the patent :-)

Ya this isn’t something that should be patentable. Tons of prior art in conference talks on this. Cool tool though, glad it’s released!

Yep. My team wrote this in 2014 or 2015, the features and conf file almost identical. We’d had to extract out of a custom Ansible module to better do the full dance.

Shared the approach with AWS ProServe, Stelligent (now Mphasis), and others as well, also discussed briefly with Atlassian architects probably in 2015-2016 timeframe.

Surprised if this isn’t one of those lots of folks just knocked out and didn’t think much of, and really should be in AWS toolkit.

Re: Peerd: an AWS VPC peering connection management tool

#27

Post by Terretta »

bmcalary_atl wrote: post #9
saber6 wrote: post #7

Earlier quoted context omitted.

Interesting concept - is this designed to be used independent of some kind of infrastructure as code tooling? I do all of my VPC peering and route-table manipulation via terraform repos today, so I am curious what the benefit of using this would be over what I have today (if there is). Thanks.

Terraform is definitely an option, but we found (the story might be different since 0.12) that creating large full meshes with many VPCs across many accounts with many route tables resulted in Terraform modules and files that were quite complex, long and hard to overlap with other meshes. This tool can be used alongside terraform, ansible or whatever infra-as-code orchestration tool you like. I encourage you to do so…

Yes, that’s the right approach. I recall we shared with Atlassian about extracting this from Ansible and keeping it outside the then brand new Terraform back when we met at a certain forum as well as when we had an NDA sync at our HQ.

It really is remarkable how identical this is to that implementation, someone using that tool would be instantly at home in this. I guess it’s one of those ‘obvious’ problems once you decide to tackle it! Well done.

I regret that back then our firm was uninterested to open source more than our enterprisification (abstraction from one CSP allowing it to work for others) of Security Monkey.

Re: Peerd: an AWS VPC peering connection management tool

#28

Post by saber6 »

bmcalary_atl wrote: post #9
saber6 wrote: post #7

Earlier quoted context omitted.

Interesting concept - is this designed to be used independent of some kind of infrastructure as code tooling? I do all of my VPC peering and route-table manipulation via terraform repos today, so I am curious what the benefit of using this would be over what I have today (if there is). Thanks.

Terraform is definitely an option, but we found (the story might be different since 0.12) that creating large full meshes with many VPCs across many accounts with many route tables resulted in Terraform modules and files that were quite complex, long and hard to overlap with other meshes. This tool can be used alongside terraform, ansible or whatever infra-as-code orchestration tool you like. I encourage you to do so…

Yup your point is true about having to manage N(N-1) number of vpc peering related components. I don’t ever deal with more than 10 VPCs at a time. Maybe you have world-conquering aspirations and need insane scale ;D
Post reply on HN