Scaling Golang CI by Replacing actions/setup-go
1–10 of 25 posts
Re: Scaling Golang CI by Replacing actions/setup-go
#2The blogpost has a lot of technical details, but you can also just read the code and try it yourself:
Re: Scaling Golang CI by Replacing actions/setup-go
#3Re: Scaling Golang CI by Replacing actions/setup-go
#4Have you considered submitting an upstream patch to the widely used actions/setup-go as well?
- Our approach writes new cache entries all the time. This can get expensive, and is a pretty big change in behavior from how actions/setup-go works today.
- actions/setup-go can basically be considered incredibly critical infrastructure for the public golang ecosystem. Any change in behavior is probably very risky and slow to happen. At this point I'd bet that we see no change, ever, in behavior.
Additionally there are a few relevant issues/prs that have been ignored for years so I'm not optimistic about contributing upstream. Frankly what we've done is write a very small bit of glue code that is likely most effective as a reference for teams writing their own custom caching actions that fit their exact needs:
- https://github.com/actions/setup-go/pull/426
- https://github.com/actions/setup-go/issues/630
- https://github.com/actions/setup-go/issues/395
- https://github.com/actions/setup-go/issues/596
That said we'd be happy if someone used our code and found it valuable! Lukas put a ton of effort into cleaning up my initial version, added the cache trimming, etc. We depend on this for all of our jobs and use it every day and think it's quite good.
Re: Scaling Golang CI by Replacing actions/setup-go
#5I’ve long wondered why setup-go was so slow and expensive it’s great to see improvements made.
Re: Scaling Golang CI by Replacing actions/setup-go
#6Have you considered submitting an upstream patch to the widely used actions/setup-go as well?
Re: Scaling Golang CI by Replacing actions/setup-go
#7Have you considered submitting an upstream patch to the widely used actions/setup-go as well?
Good question — we'd be happy to submit a PR, but it's not clear to me that they'd be interested. Some background: - Our approach writes new cache entries all the time. This can get expensive, and is a pretty big change in behavior from how actions/setup-go works today. - actions/setup-go can basically be considered incredibly critical infrastructure for the public golang ecosystem. Any change in behavior is probably…
Re: Scaling Golang CI by Replacing actions/setup-go
#8It only gets a brief mention, but the cache-pruning change was an interesting one. Cache accretion happens in the default actions/setup-go too, but dramatically increasing the number of cache-writes for cloudx-io/setup-go made it an actual issue.
As the cache grows, so does the time it takes to load it from GitHub's actions cache... and that grows until it's a significant time-suck in CI. We prune with basic mark-and-sweep.
Digging deeper, the pluggable `GOCACHEPROG` (introduced in Go 1.24) is a really useful tool. Shimming the normal cache logic for measurement, for example. In theory this should also be attractive for remote caching.
Re: Scaling Golang CI by Replacing actions/setup-go
#9However, other people around me are fine with apt installs and pip installs from global mirrors in every CI run. So I may be just autistic.