No they don't, but if you do something silly like spin up a bare VM build node, and then pull your environment every time, then obviously you won't get any caching.
Github Actions? Gitlab CI? Many of those don't do any caching because of the chance of poisoning a tag in the local registry, and don't have a good way to do caching per project. I can imagine that this affects those sorts of operations.
Yeah, that's what I was getting at - in CI I can see it, but in local dev it would seem surprising to need more than the free-with-authentication quota.
That said, if your CI needs more, it's probably time to invest in a paid account if you can, and certainly if you're commercial.
I setup a local docker cache using a docker image. The transition to HTTPS everywhere makes caching this sort of thing difficult. One has to install and often manually configure trusted certificates on every client to maximize the cache.
APK and APT caches often come hand-in-hand for this sort of thing as the logical next step is to add some OS packages to the pulled image. This also benefits from local caching and also means frustrating cache setup, certificate setup, etc.
To maximize local caching there's a lot of manual work to setup a house-of-cards series of proxies that only work on the network. Setting it up on a laptop then traveling means everything breaks in not-so-obvious ways when you leave the network.
I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…
Yes Docker seems to be struggling as a company. But I doubt Podman has anything to do with it. The adoption of Docker open-source tools is massive and 99% of its users have never heard of podman or any other clones, and likely never will. The problem is simply that those tools are free, and Docker has failed to convert the success of their free tools into a successful business.
I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…
They won the container war but lost the orchestration war. Even if docker compose was successful though, I fail to see how the clouds wouldn’t just replicate everything. So I guess they just failed to monetize the technology.
> So I guess they just failed to monetize the technology.
Yes, it's really that simple. All those "container wars" and "orchestration wars" are a distraction from the core issue, which is that all those container and orchestration tools are open-source, and it's very hard to build a viable business on top of them. Docker tried and failed, like most startups involved.
There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> This is significant - if your organization only uses a few dozen base images from DockerHub, those images will only be downloaded by each build node _once_, then never again.
Only if your build nodes have unlimited storage. If the build nodes are spun up on demand or have housecleaning tasks to prevent Tragedy of the Commons disk exhaustion, this is not true.
On the other hand, this is what caching proxies/registries are for.
> While I agree that this is the way it's supposed to work, I have unfortunately worked at companies with "stateless" build/CI servers that download the Docker image each build. Couldn't they remain stateless but be redirected through a caching proxy? Memoization is not contrary to statelessness.
Sure, now they have to build a proxy...
Artifactory is less bad than most of the tools I have to use all day.
I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…
> Redhat made Podman which is a compatible replacement I would actually prefer if they made an incompatible replacement. Docker's CLI is pretty bad in my opinion. I want to use Docker the same way I use a headless virtual machine running an SSH server. I want starting/exiting containers to be independent from their 'main process'. I want to attach/detach whenever I need to and execute arbitrary processes. -- Just use…
They made podman as a fully compatible replacement so people could easily drop-in replace their use of docker with podman, which worked.
To handle spurious interrupts from /bin/bash you can put a small script as the entrypoint containing a while true loop with a sleep infinity in it.
> Imagine if NPM, Ruby Gems, and so on rate limited package downloads until you paid! Sounds like an entirely reasonable thing to start imagining. Reliability, safety, determinism and predictability are not thrust upon someone from the commons. I frankly find it someone atrocious and abusive that downstream systems do not adequately cache these assets. The main archive repositories should be the source of truth, but…
Do you think NPM, Ruby Gems, or others would last long if they were so user hostile? Would Node have grown into the genuinely useful development environment it is if free users were limited to 10000 package downloads an hour? It's so unbelievably user hostile that it seems like the result will be people just stop using Docker. The right solution for Docker is probably to spin off or monetize the Hub in a different wa…
I think npm and ruby gems are quite a lot smaller than the average and upper bound size of docker images, which can easily be multiple GiB. Also the use case is different, where a docker build may incur a few manifest retrievals from the registry, while an npm build may incur hundreds. I don't know how quickly the average user would run into a 10000 download/hour limit for NPM, but if they were able to arrive at an equivalently high limit like 200/hr for docker, then maybe it would be fine to start charging for rate limits above that, and not impact most people.
What is a solution for the scenarios you have described? Amazon has ECR but it doesn’t support signing and doesn’t function as a proxy so you would miss upstream changes unless someone pushed them. Anything self hosted that supplies that functionality?
Pay $5.00 a month. Docker is a business that deserves to get paid if they offer something valuable.
Well that takes all the fun out of it. Looks like Docker itself has a Dockerhub proxy -
If using Alpine, looks like docker-registry is the needed package and /usr/bin/docker-registry serve /etc/docker-registry/config.yml is the command line. Next to last link has information on the config file.