21–30 of 59 posts
Congrats on shipping! We built something similar internally. Tweaking it for the right cost/availability/speed was interesting, but we now have it working to where workers are generally spun up from 0 faster than GitHub's own are.
Earlier quoted context omitted.
Yep, it's a good question! At the moment, my thoughts are roughly: GitHub's incentives and design constraints are different than ours. GitHub needs to offer something that covers a very large user-base, to cover the widest possible number of workflows, and they've done this by offering basic ephemeral VMs on-demand. CI and builds are also not GitHub's primary focus as an org. We're trying to be the absolute fastest p…
Good reply. It seems like you understand the market and where your product fits, which is half the battle. Wishing you much success.
do you actually get the promised 12.5 Gbps? I've been doing some experiments and it's really hard to get over 2.5Gbit/s upstream from AWS EC2, even when using large 64 vCPU machines. Intra-AWS (e.g. VPC) traffic is another thing and that seems to be ok.
> - Each instance has high-throughput networking of up to 12.5 Gbps, hosted in us-east-1, so interacting with artifacts, cache, container registries, or the internet at large is quick. do you actually get the promised 12.5 Gbps? I've been doing some experiments and it's really hard to get over 2.5Gbit/s upstream from AWS EC2, even when using large 64 vCPU machines. Intra-AWS (e.g. VPC) traffic is another thing and th…
- For instances with >= 32 vCPUs, traffic to an internet gateway can use 50% of the throughput
- For instances with - Traffic inside the VPC can use the full throughput
So for us, that means traffic outbound to the public internet can use up to 5 Gbps, but for things like our distributed cache or pulling Docker images from our container builders, we can get the full 12.5 Gbps.
[0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-inst...
I recently set up AWS Github runners with this terraform. It works well and you don't have to pay any extra in addition to AWS. https://github.com/philips-labs/terraform-aws-github-runner
Definitely linking OP to my team now.
How does it compare to BuildJet?
We're both offering managed GitHub Actions runners - some of the differences include: - Depot runners are hosted in AWS us-east-1, which has implications for network speed, cache speed, access to internet services, etc. (BuildJet is hosted in Europe - maybe Hetzner?) - Also thanks to AWS: each runner has a dedicated public IP address, so you're not sharing any third-party rate limits (e.g. Docker Hub) with other user…
> - Each instance has high-throughput networking of up to 12.5 Gbps, hosted in us-east-1, so interacting with artifacts, cache, container registries, or the internet at large is quick. do you actually get the promised 12.5 Gbps? I've been doing some experiments and it's really hard to get over 2.5Gbit/s upstream from AWS EC2, even when using large 64 vCPU machines. Intra-AWS (e.g. VPC) traffic is another thing and th…
We do get the promised throughput, but it depends on the destination as you've discovered. AWS actually has some docs on this[0]: - For instances with >= 32 vCPUs, traffic to an internet gateway can use 50% of the throughput - For instances with - Traffic inside the VPC can use the full throughput So for us, that means traffic outbound to the public internet can use up to 5 Gbps, but for things like our distributed c…
Are you building your base image from the GitHub runner-images repo?
Do you have any appetite for building self hosted EC2 agents for Azure DevOps and GitHub?
I'm happy to help if you are, I'm working on something similar myself for my employer.
How will you compete if GitHub talks to the Azure folks (who have the benefit of Azure scale) and gets better compute and network treatment for runners? Or is the assumption GH running remains perpetually stunted as described (which is potentially a fair and legit assumption to make based on MS silos and enterprise inertia)? To be clear, this is a genuine question, as compute (even when efficiently orchestrated and a…
Yep, it's a good question! At the moment, my thoughts are roughly: GitHub's incentives and design constraints are different than ours. GitHub needs to offer something that covers a very large user-base, to cover the widest possible number of workflows, and they've done this by offering basic ephemeral VMs on-demand. CI and builds are also not GitHub's primary focus as an org. We're trying to be the absolute fastest p…