Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

warpbuild.com

Post reply on HN
Print view View original

51–60 of 89 posts

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#51

Post by lionkor »

suryao wrote: post #49
lionkor wrote: post #48

Hi, quick question: > Runners are assigned to hardware that is ideal for build workloads with [...] high single-core performance In my kind of projects (C++, Rust, C) the builds are highly parallelizable, so single core performance is generally not what you want, if you can instead get a lot of cores. The main bottleneck to my own build pipelines on github was how painful it is to use containers, and how "helpful to…

We do have high core options too (up to 16) but not crazy high like with GPUs. You'll probably still see good benefits. Could you elaborate on the pain points with using containers?

Yes! everything changes when using containers, whereas in gitlab everything is based on containers to begin with

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#52

Post by rbultje »

suryao wrote: post #12
nwienert wrote: post #7

It’s a great idea. I’d want even faster though, GitHub Actions are quite a bit slower than my Air M2. If you spun up a fleet of top of the line Hetzner boxes I’d expect it’d be 100% faster than actions. 30% faster for half cost is just a bit too small of a gain to make the leap.

There are 2 factors here: 1) I was being conservative on my promises. For instance, we have users who reported GHA runtimes going down from ~25m to ~9m [1] 2) Local builds have the amazing advantage of being able to cache subsequent runs. CI workflows are ephemeral. This introduces a performance penalty. However, we are working on something that automagically caches some builds, especially container builds, that woul…

I've been using github actions cache to store build artifacts (object files) between builds. It takes a bit of fiddling but it's possible.

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#54

Post by yjftsjthsd-h »

solardev wrote: post #8
jeffbee wrote: post #2

How do people rationalize using a service like this for anything other than toy projects? Sending your source code to some service, then adopting and executing the artifacts it produces, means this is the central, most critical aspect of your security story. For real projects it doesn't stand even a moment's scrutiny.

A lot of smaller teams/companies don't really care about security, sadly... and is this really that different from Github Actions, CircleCI, Vercel, anything...?

It has the difference of not having been around for as long and accruing reputation. That's nothing inherent, and will come with time of course. Of those, the only other difference is that github actions has the advantage of being first party; if you use github, then there's no increase in security exposure by using their own actions.

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#55

Post by spondyl »

Oh neat, I came across BuildJet the other day.

I was trying to cross-compile a side project (https://github.com/marcus-crane/october) for Linux arm64 but trying to do so would throw up some instruction set errors.

I had parted the idea of supporting Linux arm since Github has no runners but I threw in BuildJet and it spat out a working build with no problems!

Given it only needs to run on release, for a small open source project, being charged something like 1 cent per build is surprisingly reasonable compared to having no runner at all / having to spin up a self-hosted runner :)

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#56

Post by suryao »

spondyl wrote: post #55

Oh neat, I came across BuildJet the other day. I was trying to cross-compile a side project ( https://github.com/marcus-crane/october ) for Linux arm64 but trying to do so would throw up some instruction set errors. I had parted the idea of supporting Linux arm since Github has no runners but I threw in BuildJet and it spat out a working build with no problems! Given it only needs to run on release, for a small open…

Check us out! We put some effort into the onboarding to make it super easy to use WarpBuild runners.

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#57

Post by Cu3PO42 »

suryao wrote: post #15
yjftsjthsd-h wrote: post #9

> The runners are allocated on VMs, not containers. This provides faster performance What? I haven't benchmarked it lately, but containers should (almost?) always have less overhead and better performance than VMs (I do agree that VMs are far more flexible and let you do privileged things; it's only perf that I question)

From a first-principles perspective, the stack is as follows: VMs on warpbuild: baremetal > hypervisor for VM > runner workload Container: baremetal > (cloud VM [1]) > k8s worker node OS > containerd > container OS > runner workload This assumes containers are running on k8s, which is an okay assumption in this case. The perf penalty of using a VM is much lower. Note: if you are referring to the VM spin up time, then…

I like your product and I think VMs are a fine choice, but this representation seems somewhat inaccurate. For one: don't the VMs you launch have their own OS?

And also while containerd and potentially some Linux distro in a container are involved, they aren't really adding runtime overhead. containerd (via runc) instructs the Linux kernel to isolate the workload processes, but doesn't really sit in between them and the kernel. Further, the OS in the container doesn't have its own kernel and most of the time not even its own init. It's really just a set of libraries and binaries.

I believe that you run workloads faster than competitors relying on containers, but it doesn't seem to me that containers are the problem. If you installed Linux on the same baremetal host, I'm convinced you can get that same performance in a container that you can get in a VM on that host.

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#58

Post by thaeli »

rbultje wrote: post #52
suryao wrote: post #12

Earlier quoted context omitted.

There are 2 factors here: 1) I was being conservative on my promises. For instance, we have users who reported GHA runtimes going down from ~25m to ~9m [1] 2) Local builds have the amazing advantage of being able to cache subsequent runs. CI workflows are ephemeral. This introduces a performance penalty. However, we are working on something that automagically caches some builds, especially container builds, that woul…

I've been using github actions cache to store build artifacts (object files) between builds. It takes a bit of fiddling but it's possible.

Yeah, interested to compare this with optimized and properly cached GitHub Hosted Runner builds.

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#59

Post by suryao »

Cu3PO42 wrote: post #57
suryao wrote: post #15

Earlier quoted context omitted.

From a first-principles perspective, the stack is as follows: VMs on warpbuild: baremetal > hypervisor for VM > runner workload Container: baremetal > (cloud VM [1]) > k8s worker node OS > containerd > container OS > runner workload This assumes containers are running on k8s, which is an okay assumption in this case. The perf penalty of using a VM is much lower. Note: if you are referring to the VM spin up time, then…

I like your product and I think VMs are a fine choice, but this representation seems somewhat inaccurate. For one: don't the VMs you launch have their own OS? And also while containerd and potentially some Linux distro in a container are involved, they aren't really adding runtime overhead. containerd (via runc) instructs the Linux kernel to isolate the workload processes, but doesn't really sit in between them and t…

You're right - I missed that VM OS in my illustration up there.

Benchmarks are hard to come by but iirc, each VMM adds a 3-5% overhead. The gatekeeping for permissions and sandboxing that is done at each level involves active compute cycles and that adds a little overhead.

Something of the order of 10% may not be large but there's a difference.[1]

Disk IO is another major factor btw, since virtual filesystems can be ... flaky.

[1] https://www.vmware.com/pdf/hypervisor_performance.pdf

Re: Show HN: WarpBuild – x86-64 and arm GitHub Action runners for 30% faster builds

#60

Post by xjia »

btown wrote: post #13

Congrats on the launch! There do seem to be a number of other entrants in this space: https://github.com/neysofu/awesome-github-actions-runners#li... What makes you stand out from the pack? The VM approach seems very cool - is this unique in the space? Do you have different approaches that provide speedups or security benefits not possible with other third party runner systems? Any benchmarks against competitors? Sep…

We are essentially competing in the same space with https://dime.run/

We also use VMs. But they are persistent. So you always see your runners as Online in GitHub UI.

We achieve this by investing in virtualization technology so that idle runner VMs do not consume too many resources. Disclaimer: I used to work for Google Cloud.

Post reply on HN