Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker to rate limit image pulls

docker.com

Post reply on HN
Print view View original

231–240 of 274 posts

Re: Docker to rate limit image pulls

#231

Post by terom »

eblanshey wrote: post #104
jopsen wrote: post #59

Earlier quoted context omitted.

Sure, now they have to build a proxy...

Super easy to run a docker cache proxy: docker run -d -p 6000:5000 \ -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \ --restart always \ --name registry registry:2 That's it. Now fetch docker images from the IP that command is running on. Taken from gitlab: https://docs.gitlab.com/runner/install/registry_and_cache_se...

Is that going to actually help with the manifest-based rate limits? It sounds like it only caches the layers, the manifest metadata for a tag is not cached.

https://docs.docker.com/registry/recipes/mirror/#what-if-the...

> When a pull is attempted with a tag, the Registry checks the remote to ensure if it has the latest version of the requested content. Otherwise, it fetches and caches the latest content.

Re: Docker to rate limit image pulls

#232

Post by EnigmaCurry »

efreak wrote: post #188
hamiltont wrote: post #110

Earlier quoted context omitted.

Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid

Personally, I wish generic caching proxies were still a thing, and easier to set up. I've tried setting up squid several times in the past, and failed miserably every single time--all I want to do is use it as a gateway (ie, make the proxy invisible to the application) for e.g. apt packages, so I just ended up using apt-cache or whatever other appropriate software, but I'd far rather use something generic that just w…

this is my way: https://github.com/EnigmaCurry/lazy-distro-mirrors

Re: Docker to rate limit image pulls

#233

Post by efrecon »

mschuster91 wrote: post #136
hamiltont wrote: post #110

Earlier quoted context omitted.

Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid

Just drop a Sonatype Nexus instance on a Docker container somewhere on your network. Alternatively, use Squid if you don't push to the public Docker registry, although you might need to mess around with internal CA for SSL...

I would stay away from Nexus. It has problems with latest tags.

Re: Docker to rate limit image pulls

#234

Post by neurostimulant »

threeseed wrote: post #225
manquer wrote: post #213

Earlier quoted context omitted.

Just to add all the major cloud service providers provide registries ACR /ECR/GCR etc . If you run k8s service with one of the them in my experience it is best to use the corresponding registry. I have pulled and run 20k times a 1GB image in less than 10-15 minutes without breaking a sweat. Finally GitHub packages offers a registry out of the box . It is great for CI and devs to access . I generally have the tags mir…

Github Docker Registry is a mess and should be avoided at all costs. 1) It is broken and unusable on Kubernetes and Docker Swarm. 2) It is flaky often returning 500 type errors. 3) It is expensive as the amount of pull bandwidth is very limited.

> It is broken and unusable on Kubernetes and Docker Swarm.

Hmm, I use them on several kubernetes clusters in the past few months and don't see any issue yet.

Re: Docker to rate limit image pulls

#235

Post by guillaumei »

cddotdotslash wrote: post #131
msla wrote: post #109

Earlier quoted context omitted.

$5/month isn't $5/month. It's convincing your boss you need $5/month, because they need to convince their boss, which eventually makes its way up the chain to C-levels, who don't know Docker from yesterday's rotting tuna casserole and view eating either that or the $5/month with the same level of disdain. It isn't about the money, it's about the Mommy-May-I up and down the chain with emails and meetings and careful e…

If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

It took 3 months at my old client (fortune 100) to get a signature for an Addendum that would deliver us a service for free in addition to the paid stuff that was already signed.

Re: Docker to rate limit image pulls

#236

Post by skocznymroczny »

mm89 wrote: post #119
temikus wrote: post #105

Earlier quoted context omitted.

It may be general resistance to a concept. More and more services are becoming subscription based, so 5$ here, 5$ there and the cost creeps further.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

Providing objective, non-biased events would be very hard.

Consider for example the current riots going on in the US. How do you objectively report on that? With bias, on one side you have "peaceful protest disrupted and escalated by the police", on the other side you have "police intervening in riots to maintain order and protect property". There's not really inbetween.

Re: Docker to rate limit image pulls

#237

Post by majewsky »

swuecho wrote: post #223
radlad wrote: post #197

Earlier quoted context omitted.

Isn't there no authentication on that registry? I guess that's fine if you don't believe in zero-trust architecture.

you are right. That is what you can get in minutes.

If you cannot get a TLS cert for internal infrastructure in a few minutes, I'd recommend you start looking into why.

Re: Docker to rate limit image pulls

#238

Post by majewsky »

skocznymroczny wrote: post #236
mm89 wrote: post #119

Earlier quoted context omitted.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

Providing objective, non-biased events would be very hard. Consider for example the current riots going on in the US. How do you objectively report on that? With bias, on one side you have "peaceful protest disrupted and escalated by the police", on the other side you have "police intervening in riots to maintain order and protect property". There's not really inbetween.

"Protesters say their peaceful assembly has been disrupted and escalated by the police. The police argue they've only been intervening in riots to maintain order and protect property."

Done.

Re: Docker to rate limit image pulls

#239

Post by [deleted] »

nrmitchi wrote: post #38
bsamuels wrote: post #14

There is very little reason for a build node to need to pull 200 images in 6 hours, and here is why: When a machine issues a ``docker build`` command, the program reads the relevant dockerfile to check for any base images that need to be pulled (a la "FROM:") These base images are identified based on the image repository, image name, and image tag. The first thing docker does is it checks its local registry and tries…

I don't disagree with what you are saying, _but_: > Docker defines pull rate limits as the number of manifest requests to Docker Hub. > For example, if you already have the image, the Docker Engine client will issue a manifest request, realize it has all of the referenced layers based on the returned manifest, and stop. ... ... So an image pull is actually one or two manifest requests, This still implies that even if…

[deleted]

Re: Docker to rate limit image pulls

#240

Post by prepend »

pjmlp wrote: post #228
ironmagma wrote: post #161

Earlier quoted context omitted.

Except it’s not $5, it’s $5 to sign up, then an email and a phone call and your firstborn dog to unsubscribe. If it were microtransactions, that’d be one thing...

If the service doesn't play ball there is always the consumer protection agency.

That doesn’t work. I find it better to stay anonymous and avoid spam from these services.

The perpetual spam is worse for me than the $5.

Post reply on HN