They want you to login, so they have better data for monetization
Docker to rate limit image pulls
11–20 of 274 posts
Re: Docker to rate limit image pulls
#12I'm very conflicted about this. On the one hand, I recognize that there are potentially significant costs to be born to serve these repositories. On the other hand, making docker part of your infrastructure requires a certain degree of availability. At some level this seems to me like using my IDE and after 6 hours it would stop working or finding that my CDNJS references to bootstrap stopped working after 6 hours of…
Do pulls for a "latest" tag bypass the cache?
Re: Docker to rate limit image pulls
#13I'm very conflicted about this. On the one hand, I recognize that there are potentially significant costs to be born to serve these repositories. On the other hand, making docker part of your infrastructure requires a certain degree of availability. At some level this seems to me like using my IDE and after 6 hours it would stop working or finding that my CDNJS references to bootstrap stopped working after 6 hours of…
It really sounds like a problem we just have to solve. Putting all the expectation on one central service is not reliable.. it should be a distributed network of content delivery nodes, of which you run a few yourself.
Re: Docker to rate limit image pulls
#14There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why:
When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:")
These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries to find a match for the base image the docker build is requesting. If a matching image is located in the local registry, it uses that one in lieu of downloading the image.
This is significant - if your organization only uses a few dozen base images from DockerHub, those images will only be downloaded by each build node _once_, then never again.
Many docker users erroneously believe that if their Dockerfile requests a "latest" tagged image, docker build will always download the newest version of the image. However, the "latest" tag is literally just a tag, it doesn't have any special functionality built in. If the docker build command finds an image tagged "latest" in the local registry, it stops there.
The only way to get docker build to always use the "actual latest" version of the base image is to add the "--pull" parameter to the docker build command. This arg will tell docker build to check the repository remote to see if the SHA hash of the image tagged "latest" has changed, and if so, re-download and use it. In the absolute worst case, this means each build node will pull 1 copy of each base image when the base image is updated. So unless you use 200 different base images that all have updates deployed to Dockerhub each and every day, you are fine.
Re: Docker to rate limit image pulls
#15Good!
I'd even welcome much more agressive limits than what they're proposing; the current culture regarding builds and CI in general is horrifyingly ineffficient, wasteful and in the end just plain slow.
I'm looking forward to developers adjusting their workflows (and caches, etc.) to actual, reasonable limits, not just perusing the service as if it were an unlimited cost-free cornucopia of software.
Re: Docker to rate limit image pulls
#16Can’t say I’m surprised. They must be incurring significant data costs. The limits seem reasonable at least.
Re: Docker to rate limit image pulls
#17There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> The first thing docker does is it checks its local registry and tries to find a match for the base image the docker build is requesting. If a matching image is located in the local registry, it uses that one in lieu of downloading the image.
While I agree that this is the way it's supposed to work, I have unfortunately worked at companies with "stateless" build/CI servers that download the Docker image each build.
Re: Docker to rate limit image pulls
#18There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> The first thing docker does is it checks its local registry and tries to find a match for the base image the docker build is requesting. If a matching image is located in the local registry, it uses that one in lieu of downloading the image. While I agree that this is the way it's supposed to work, I have unfortunately worked at companies with "stateless" build/CI servers that download the Docker image each build.
Well, this policy change will force them to be more efficient, and it's a net win for everyone
Re: Docker to rate limit image pulls
#19Honest question, but why can't docker use something like bittorrent to download images?
Most of us download our OS via torrents only, so we may as well download the images too if there was support for it.
Re: Docker to rate limit image pulls
#20I’m hoping GCP/AWS steps up and creates a limitless docker registry. Considering the size of their infrastructure operations, I suspect this would be a small cost but bring a lot of goodwill. One thing this is bound to do is to make the process of using docker a bit more complex. Explicit registries will probably start to be used everywhere, which is something I welcome. But it seems like a really poor decision by do…
They have registries already which work with docker after an auth setup. They are generally private to you.
Yes, docker is a struggling company which sold some lines of business and is now trying to reinvent itself again towards developers. Given other recent moves, I'm not sure the new leadership understands how to do this.