I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…
Docker to rate limit image pulls
31–40 of 274 posts
Re: Docker to rate limit image pulls
#32Honest question, but why can't docker use something like bittorrent to download images? Most of us download our OS via torrents only, so we may as well download the images too if there was support for it.
> Most of us download our OS via torrents only I mean, I only do it to stick it to the people who claim torrents can only be used for piracy; I think most people prefer the simplicity of direct downloads though…
Re: Docker to rate limit image pulls
#33Honest question, but why can't docker use something like bittorrent to download images? Most of us download our OS via torrents only, so we may as well download the images too if there was support for it.
The hardest part of that would be verifying image authenticity. Google Cloud uses an adjacent feature called binary authorization. When turned on, only images that are signed by a given authority (usually your ci/cd instruments) can be run inside your Kubernetes cluster. Binary authorization may be a good starting point for someone trying to make bittorrent distributed images a usable thing.
Re: Docker to rate limit image pulls
#34Earlier quoted context omitted.
> Most of us download our OS via torrents only I mean, I only do it to stick it to the people who claim torrents can only be used for piracy; I think most people prefer the simplicity of direct downloads though…
I can usually max out my connection speed with torrents. This is rarely the case with direct downloads.
Re: Docker to rate limit image pulls
#35There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> The first thing docker does is it checks its local registry and tries to find a match for the base image the docker build is requesting. If a matching image is located in the local registry, it uses that one in lieu of downloading the image. While I agree that this is the way it's supposed to work, I have unfortunately worked at companies with "stateless" build/CI servers that download the Docker image each build.
Couldn't they remain stateless but be redirected through a caching proxy? Memoization is not contrary to statelessness.
Re: Docker to rate limit image pulls
#36Earlier quoted context omitted.
The hardest part of that would be verifying image authenticity. Google Cloud uses an adjacent feature called binary authorization. When turned on, only images that are signed by a given authority (usually your ci/cd instruments) can be run inside your Kubernetes cluster. Binary authorization may be a good starting point for someone trying to make bittorrent distributed images a usable thing.
> The hardest part of that would be verifying image authenticity. That's exactly what Bittorrent does with its hash tree. You'd get the root hash (extremely tiny) from Docker Hub, and the rest of the metadata, as well as the data blocks, from the swarm. The authenticity is all handled by the TLS that serves you the root infohash from Docker Hub. It's a Merkle tree: the root hash is for the metadata, which is a list o…
You get the hash of the final result from the trusted server and hash is checked. Because of this you will never get an invalid image.
There are also some clever tricks to make sure no one can force you to start over from scratch by sending wrong data. But that's more of a detail.
Re: Docker to rate limit image pulls
#37There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
Re: Docker to rate limit image pulls
#38There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…
> Docker defines pull rate limits as the number of manifest requests to Docker Hub.
> For example, if you already have the image, the Docker Engine client will issue a manifest request, realize it has all of the referenced layers based on the returned manifest, and stop. ... ... So an image pull is actually one or two manifest requests,
This still implies that even if you are appropriately re-using layers on your machine, with a free plan you can only do maximum 200 builds (since docker still needs to verify it has the image) per 6 hours?
This change also seems to imply that builds steps which previously did not handle/require authentication against Docker hub (it was only pulling public images, and pushing elsewhere) will now be required to auth against docker hub in order to double the number of pulls/checks/builds it is allowed?
Re: Docker to rate limit image pulls
#39Earlier quoted context omitted.
The hardest part of that would be verifying image authenticity. Google Cloud uses an adjacent feature called binary authorization. When turned on, only images that are signed by a given authority (usually your ci/cd instruments) can be run inside your Kubernetes cluster. Binary authorization may be a good starting point for someone trying to make bittorrent distributed images a usable thing.
What is the issue here? Is the torrent checksum (provided by docker) not enough?
When the backdoor is detected, you now need a revocation system so the distribution of the malicious image will die. You can theoretically do this on the tracker level, but people may build other trackers that may not propagate the changes.
Re: Docker to rate limit image pulls
#40Honest question, but why can't docker use something like bittorrent to download images? Most of us download our OS via torrents only, so we may as well download the images too if there was support for it.
> Most of us download our OS via torrents only I mean, I only do it to stick it to the people who claim torrents can only be used for piracy; I think most people prefer the simplicity of direct downloads though…
Afaik Windows is also using this to install updates where it shares the download with others in the region (1) using p2p (though i may be wrong since i don't use Windows anymore)
(1) https://www.itproportal.com/amp/news/how-to-stop-windows-10-...