MS should just buy them and put out of their misery.
Docker to rate limit image pulls
91–100 of 274 posts
Re: Docker to rate limit image pulls
#92Earlier quoted context omitted.
> The first thing docker does is it checks its local registry and tries to find a match for the base image the docker build is requesting. If a matching image is located in the local registry, it uses that one in lieu of downloading the image. While I agree that this is the way it's supposed to work, I have unfortunately worked at companies with "stateless" build/CI servers that download the Docker image each build.
Well, this policy change will force them to be more efficient, and it's a net win for everyone
Or just pony up the $5/mo for Pro... not as fun as re-engineering your CI pipeline, once again.
Re: Docker to rate limit image pulls
#93Earlier quoted context omitted.
CI/CD systems on AWS, Azure, GCP and others might be running on Kubernetes containers (using kaniko, podman, etc) or using Docker-in-Docker, and there isn't a widely supported or in-cloud-platform tool for sharing cached layers. And as pointed out below, even if you are intelligently caching layers, manifest requests count as a pull. As far as I know, no caching proxies exist for Docker that support limiting manifest…
Surprised to see Docker-in-Docker mentioned so deeply down here. It’s an extremely valid way of doing things, and non-trivial to implement a caching layer for.
Usually docker outside of docker is used, no? If the image is cached on the host, it would be available to any container having access to the docker daemon socket as well since it's the same daemon.
Re: Docker to rate limit image pulls
#94I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…
After Docker Swarm failed it was clear that they could not survive just on the core Docker tech and CLI, which are all becoming less valuable day by day due to the various open container initiatives. In the absence of a killer product they are still a ripe acquisition target, but not a successful business.
Re: Docker to rate limit image pulls
#95Earlier quoted context omitted.
CI/CD systems on AWS, Azure, GCP and others might be running on Kubernetes containers (using kaniko, podman, etc) or using Docker-in-Docker, and there isn't a widely supported or in-cloud-platform tool for sharing cached layers. And as pointed out below, even if you are intelligently caching layers, manifest requests count as a pull. As far as I know, no caching proxies exist for Docker that support limiting manifest…
Surprised to see Docker-in-Docker mentioned so deeply down here. It’s an extremely valid way of doing things, and non-trivial to implement a caching layer for.
Isn't Docker-in-Docker actually using the host's Docker daemon? I am mounting the docker socket in all my Docker-in-Docker containers, thus all the build tasks running on the same host can share the caches.
I guess one could have docker containers that actually run docker, but I don't see a reason to do that...
Re: Docker to rate limit image pulls
#96Earlier quoted context omitted.
Well that takes all the fun out of it. Looks like Docker itself has a Dockerhub proxy - https://docs.docker.com/registry/recipes/mirror/ https://hackernoon.com/mirror-cache-dockerhub-locally-for-sp... https://stackoverflow.com/questions/32531048/docker-pull-thr... https://docs.docker.com/registry/configuration/ https://www.google.com/amp/s/ops.tips/amp/gists/aws-s3-priva... If using Alpine, looks like docker-registry…
Per your first link: > What if the content changes on the Hub? > When a pull is attempted with a tag, the Registry checks the remote to ensure if it has the latest version of the requested content. Otherwise, it fetches and caches the latest content. If that causes a manifest pull, it counts as a pull and will be rate limited. Yikes! This could lead to wildly nondeterministic behavior.
Yes, it does a pull but caches the response, so subsequent pulls should hit the local cache and not be limited.
Re: Docker to rate limit image pulls
#97Earlier quoted context omitted.
> Most of us download our OS via torrents only I mean, I only do it to stick it to the people who claim torrents can only be used for piracy; I think most people prefer the simplicity of direct downloads though…
I can usually max out my connection speed with torrents. This is rarely the case with direct downloads.
On symmetric gigabit fiber only a few services, mainly steam and battle.net have ever given me 90mbyte/s download speeds. A surprising amount will limit to 500mbit or less, even if you have the download pipe for it.
Re: Docker to rate limit image pulls
#98Re: Docker to rate limit image pulls
#99I originally came here to ask how folks use that many docker images in six hours (I'm mostly a Docker n00b, and not being facetious); however, after reading the article, I clicked to see how much unlimited is and it's $5 a month. Now my question has changed to: is $5 a month really a deal breaker for folks to get unlimited? Or what are the use cases where the cost is prohibitive? Open source or community projects?
In most cases it would seem $5/user isn't much as a business or organizational expense, and if it's a personal project 200 images in six hours seems pretty solid?
I'm just sort of shocked it's so cheap. I figured it'd be like $25-$100 a month or something just because of all the bandwidth someone could probably burn building random/broken shit over and over.
I'm mostly curious because I've considered using Docker recently for personal projects and my home server; but I'd rather not invest a bunch of time porting things only to find out it's gonna be way more than $5 (not including my time).
Or is it that hard to just build your own images? To be honest, I've never really understood that part of Docker (using other folks images)... It always seemed like an enormous security risk[1]. FWIW, I still deploy my personal projects using chef/ansible, shell scripts, and systemd units like some sort of curmudgeon-y monster...
[1] Everything is a security risk (APT, source, etc), I know, I get it. No one need scribe miles of pedantry into the comments explaining it to me. It's what each of us has tolerance for and can accept that matters.
Re: Docker to rate limit image pulls
#100There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> This is significant - if your organization only uses a few dozen base images from DockerHub, those images will only be downloaded by each build node _once_, then never again. You're assuming that the set of build nodes is relatively static. Plenty of architectures set up autoscaling for the underlying nodes, that terminate servers that aren't being used and relatively soon enough (tens of minutes, hours) spin up ne…
If you're doing this, you're in a cloud environment that's also proximate to a blob store, and can trivially host your own registry.