Google's Cloud Build has always worked very well for me for remote builds, but it'd be nice if BuildKit works as consistent service interface so it's easy to switch between build backend providers.
Faster Docker builds using a remote BuildKit instance
11–20 of 48 posts
Re: Faster Docker builds using a remote BuildKit instance
#1230 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more servi…
> My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services. If you're deploying all your "microservices" in parallel, then what you might have built is a distributed monolith. A microservice can be tested and deployed independently.
Whether it is a distributed monolith or a set of microservices is independent of the speed of build.
Re: Faster Docker builds using a remote BuildKit instance
#1330 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more servi…
> My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services. If you're deploying all your "microservices" in parallel, then what you might have built is a distributed monolith. A microservice can be tested and deployed independently.
I don't see a contradiction. I read it that the microservices are independent and thus can build in parallel, if several teams work on changes to several microservices.
Spinning a build worker outright when a change us pushed is the fastest way, and may be expensive if the build process is prolonged.
OTOH I've seen much faster image build times, with smart reuse of layers, so that you don't have to re-run that huge npm install if your packages.lock did not change.
Re: Faster Docker builds using a remote BuildKit instance
#1430 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more servi…
> 30 minute docker builds? At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.
I'm working on an ungodly pile of hacks (https://github.com/jeffparsons/hope) to help with this. Coming Soon™: S3 backend and better tests.
Re: Faster Docker builds using a remote BuildKit instance
#15Earlier quoted context omitted.
> 30 minute docker builds? At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.
I'm working on an ungodly pile of hacks ( https://github.com/jeffparsons/hope ) to help with this. Coming Soon™: S3 backend and better tests.
`hope` is a good name for a service trying to solve this problem :D
Re: Faster Docker builds using a remote BuildKit instance
#16This is fairly similar in concept to what we do over at depot.dev
https://depot.dev/blog/depot-magic-explained
We've found that BuildKit has several inefficiencies preventing it from being as fast as it could be in the cloud, especially when dealing with simultaneous builds (common in CI). That led us to create our own optimized fork of BuildKit.
The number of fine-tuning knobs you can turn running a self-hosted BuildKit instance is limitless, but I also encourage everyone to try it as a fantastic learning exercise.
Re: Faster Docker builds using a remote BuildKit instance
#1730 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more servi…
I regularly build images where we install Python from source, which makes 30m seem quite normal...
Re: Faster Docker builds using a remote BuildKit instance
#1830 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more servi…
I regularly build images where we install Python from source, which makes 30m seem quite normal...
Why?
Re: Faster Docker builds using a remote BuildKit instance
#19Earlier quoted context omitted.
I'm working on an ungodly pile of hacks ( https://github.com/jeffparsons/hope ) to help with this. Coming Soon™: S3 backend and better tests.
`hope` is a good name for a service trying to solve this problem :D
Haha, thanks. I chose it because:
- Here's One (I? You?) Prepared Earlier - Sometimes Hope _is_ a (caching) strategy - And yeah, I really hope I can make this thing work well.
I like silly puns. They bring me joy.
Re: Faster Docker builds using a remote BuildKit instance
#20This is pretty cool - provides a good speed up for container builds. The couple of beefy instances can set you back $200-1000 a month on aws apart from the regular github action runner costs and it only goes up from there. We have a way around that plus effective scaling for multiple parallel builds with WarpBuild.
As a side note: In my time running a CI infra co, we see that a majority of the workflow time for large teams comes from tests - which can have over 200 shards in some cases.