Show HN: Burrow is a globally distributed, serverless HTTP proxy
1–10 of 13 posts
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#2Very cool. Is there a reason to choose AWS lambda over other tech (say Cloudflare workers)?
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#3Very cool. Is there a reason to choose AWS lambda over other tech (say Cloudflare workers)?
Cloudflare Workers are great in general. The way they route client requests to the nearest Cloudflare data center means that a single client will most often end up with a small number (perhaps even 1) of public IP addresses coming out of Cloudflare. And to my knowledge, there's not a way to ask for your client to be routed in a different way. So due to this, AWS Lambda's inherent region-specific behavior is actually an advantage for Burrow, since we can ask to be routed through a specific region (no matter how distant) by virtue of hitting the per-region Lambda URLs.
That said, you could actually layer the two together if you were so inclined, and/or optionally route through Cloudflare and AWS.
Making the Cloudflare Worker implementation of this would be trivial and there's no dependency on Go in any way for the proxy itself.
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#4[deleted]
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#5The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal? It also seems problematic for re-use since not all accounts have those regions enabled. Perhaps using something like the below to discover enabled regions at deploy time would make it more "consumable" by other folks:
aws account list-regions \
--region-opt-status-contains "ENABLED" "ENABLED_BY_DEFAULT" \
--no-paginate \
--query "Regions[].RegionName" \
--output text
[1] https://github.com/mlhpdx/email-origin/blob/main/scripts/dep...
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#6The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal? It also seems problematic for re-use since not all accounts have those regions enabled. Perhaps using something like the below to discover enabled regions at deploy time would make it more "consumable" by other folks: aws account list-regions \ --region-opt-status-contains "ENABLED" "ENABLE…
Aha, indeed. A few interesting notes on this. And thanks for the link to your Makefile.
I see you're a Cloudformation Stacks user (good stuff), so you may know some of what I say below already, but maybe the Terraform specific notes will be of interest.
Many AWS deployments (Terraform or otherwise) are single region unfortunately, with a primary factor there being that AWS APIs are for the most part region-specific. The AWS Terraform provider was built with this mindset too, unfortunately. Then the situation is complicated by Terraform's lack of support for dynamic providers, which is one of the most discussed still-open issues in Terraform.
Some related open issues:
https://github.com/hashicorp/terraform/issues/24476 https://github.com/hashicorp/terraform/issues/25244 https://github.com/hashicorp/terraform/issues/19932
My understanding is that people that have this as a first-class concern use something like Terragrunt to dynamically generate the HCL as needed.
https://terragrunt.gruntwork.io/
For Burrow here, I was was focused on shipping something and didn't want to spend time on generating that code just yet. But the project has gotten some attention so if this is a recurring request I'll probably do it.
Note that I intentionally chose the 17 AWS regions that are default enabled in every new AWS account, so it should only be if you want to customize it that this becomes important.
A Cloudformation Stacks based deployment to achieve multi-region would be another good (and perhaps better) option here. I could add that to the Makefile, with inspiration from your link there.
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#7The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal? It also seems problematic for re-use since not all accounts have those regions enabled. Perhaps using something like the below to discover enabled regions at deploy time would make it more "consumable" by other folks: aws account list-regions \ --region-opt-status-contains "ENABLED" "ENABLE…
Aha, indeed. A few interesting notes on this. And thanks for the link to your Makefile. I see you're a Cloudformation Stacks user (good stuff), so you may know some of what I say below already, but maybe the Terraform specific notes will be of interest. Many AWS deployments (Terraform or otherwise) are single region unfortunately, with a primary factor there being that AWS APIs are for the most part region-specific.…
Oh I realized I was thinking of Cloudformation Stacksets, since those like you automate deployment of a Stack across multiple regions.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGui...
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#8The amount of duplication in `main.tf` (copy/pasta for each region) is shocking, but I'm not a Terraform user -- is that normal? It also seems problematic for re-use since not all accounts have those regions enabled. Perhaps using something like the below to discover enabled regions at deploy time would make it more "consumable" by other folks: aws account list-regions \ --region-opt-status-contains "ENABLED" "ENABLE…
Yea, unfortunately dynamic provider blocks to use a loop to handle this type of thing have been on the wishlist forever. IIRC Hashicorp's response is that it simply isn't possible, but I believe the OpenTofu project was targeting this (or may have already done it).
I manage a particular file right now that has 100 entries of something very similar. Just silly and annoying to search through and prone to copy/paste errors (one time I accidentally deployed something to a prod account in this way).
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#9Very cool. Is there a reason to choose AWS lambda over other tech (say Cloudflare workers)?
Cloudflare Workers are great in general. The way they route client requests to the nearest Cloudflare data center means that a single client will most often end up with a small number (perhaps even 1) of public IP addresses coming out of Cloudflare. And to my knowledge, there's not a way to ask for your client to be routed in a different way. So due to this, AWS Lambda's inherent region-specific behavior is actually…
Indeed. As another example, I deploy regional API Gateways for "same region" access to service stacks and then tie them into the "global" service using Route53 latency based routing records (one per region). It works nicely to get the best of both worlds.
Re: Show HN: Burrow is a globally distributed, serverless HTTP proxy
#10Earlier quoted context omitted.
Cloudflare Workers are great in general. The way they route client requests to the nearest Cloudflare data center means that a single client will most often end up with a small number (perhaps even 1) of public IP addresses coming out of Cloudflare. And to my knowledge, there's not a way to ask for your client to be routed in a different way. So due to this, AWS Lambda's inherent region-specific behavior is actually…
Indeed. As another example, I deploy regional API Gateways for "same region" access to service stacks and then tie them into the "global" service using Route53 latency based routing records (one per region). It works nicely to get the best of both worlds.
Nice. I was wondering if r53 latency-based routing would work as an optional add-on to this. I didn't look into it but was thinking it might not work because the latency based routing records are A records that have to point at a fixed IP address right (rather than the function url in my case). But presumably API gateway just gives you hostnames as well so maybe I'm mistaken here?