Cache is King: A guide for Docker layer caching in GitHub Actions
1–10 of 111 posts
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#2this is pretty neat—it’s been a while since i’ve tried caching layers with gha. it used to be quite frustrating.
my previous experience was that in nearly all situations the time spent sending and retrieving cache layers over the network wound up making a shorter build step moot. ultimately we said “fuck it” and focused on making builds faster without (docker layer) caching.
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#3I weep for this period of time where we don't have sticky disks readily available for builds. Uploading the layer cache each time is such a coarse and time-consuming way to cache things.
Maybe building from scratch all the time is a good correctness decision? Maybe stale values in disks is a tricky enough issue to want to avoid entirely?
If you keep a stack of disks around and grab a free one when the job starts you'd end up with good speedup a lot of the time. If cost is an issue you can expire them quickly. I regularly see CI jobs spending >50% of their time downloading the same things, or compiling the same things, over and over. How many times have I triggered an action that compiled the exact same sqlite source code? Tens of thousands?
Maybe this is fine, I dunno.
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#4I weep for this period of time where we don't have sticky disks readily available for builds. Uploading the layer cache each time is such a coarse and time-consuming way to cache things. Maybe building from scratch all the time is a good correctness decision? Maybe stale values in disks is a tricky enough issue to want to avoid entirely? If you keep a stack of disks around and grab a free one when the job starts you'…
I agree. The notion that everything must be docker is nice in principle but requires a lot of performance optimization work early on. Earlier than one would need with "sticky disks" as you called them.
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#5I weep for this period of time where we don't have sticky disks readily available for builds. Uploading the layer cache each time is such a coarse and time-consuming way to cache things. Maybe building from scratch all the time is a good correctness decision? Maybe stale values in disks is a tricky enough issue to want to avoid entirely? If you keep a stack of disks around and grab a free one when the job starts you'…
This is exactly the sort of insight that led us to work on Blacksmith. Since we own the hardware we run CI jobs on there are some exciting things we can do to make these "sticky disks" work the way you describe it. Stay tuned!
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#6I weep for this period of time where we don't have sticky disks readily available for builds. Uploading the layer cache each time is such a coarse and time-consuming way to cache things. Maybe building from scratch all the time is a good correctness decision? Maybe stale values in disks is a tricky enough issue to want to avoid entirely? If you keep a stack of disks around and grab a free one when the job starts you'…
[deleted]
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#7this is pretty neat—it’s been a while since i’ve tried caching layers with gha. it used to be quite frustrating. my previous experience was that in nearly all situations the time spent sending and retrieving cache layers over the network wound up making a shorter build step moot. ultimately we said “fuck it” and focused on making builds faster without (docker layer) caching.
Yeah that still holds true to some extent today with the GHA cache. Blacksmith colocates its cache with our CI runners, and ensures that they're in the same local network allowing us to saturate the NIC and provide much faster cache reads/writes. We're also thinking of clever ways to avoid downloading from a cache entirely and instead bind mount cache volumes over the network into the CI runner. Still early days, but stay tuned!
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#8This is wild. I've spent the last three weeks working on this stuff for two separate clients.
Important note if you're taking advice: cache-from and cache-to both accept multiple values. Cache to just ouputs the cache data to all the ones specified. cache-from looks for cache hits in the sources in-order. You can do some clever stuff to maximize cache hits with the least amount of downloading using the right combination.
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#9I feel like ive seen so many new companies just providing cheaper github actions.
Re: Cache is King: A guide for Docker layer caching in GitHub Actions
#10This is wild. I've spent the last three weeks working on this stuff for two separate clients. Important note if you're taking advice: cache-from and cache-to both accept multiple values. Cache to just ouputs the cache data to all the ones specified. cache-from looks for cache hits in the sources in-order. You can do some clever stuff to maximize cache hits with the least amount of downloading using the right combinat…
oh TIL, that is interesting