Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker to rate limit image pulls

docker.com

Post reply on HN
Print view View original

161–170 of 274 posts

Re: Docker to rate limit image pulls

#161

Post by ironmagma »

pjmlp wrote: post #132
mm89 wrote: post #119

Earlier quoted context omitted.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

Mind blowing, imagine paying $5 for each newspaper one wants to read.

Except it’s not $5, it’s $5 to sign up, then an email and a phone call and your firstborn dog to unsubscribe. If it were microtransactions, that’d be one thing...

Re: Docker to rate limit image pulls

#162

Post by alpb »

Also worth mentioning they added a 6 month retention policy for unpaid accounts. https://www.docker.com/pricing/resource-consumption-updates If you don't want your images disappearing, you should probably move off of Docker Hub. (That said, I don't think there's any reason to continue using Docker Hub, even with a paid service. I suspect such docker services will be discontinued gradually.)

It's not hard to run a registry yourself. https://docs.docker.com/registry/deploying/ Registry supports all sorts of storage backends (files, S3, GCS ...). It's just a stateless HTTP server. It's not hard to configure for production (it comes with decent defaults). If you deploy this to something like Google Cloud Run, you can get ~free hosting (sans the storage costs) for the "registry" itself plus TLS and autoscaling too (I should probably write a tutorial on this).

Re: Docker to rate limit image pulls

#163

Post by koolba »

Androider wrote: post #92
umvi wrote: post #18

Earlier quoted context omitted.

Well, this policy change will force them to be more efficient, and it's a net win for everyone

Or just pony up the $5/mo for Pro... not as fun as re-engineering your CI pipeline, once again.

You have to re-engineer it anyway to authenticate your Pro account.

Re: Docker to rate limit image pulls

#164

Post by manigandham »

mm89 wrote: post #119
temikus wrote: post #105

Earlier quoted context omitted.

It may be general resistance to a concept. More and more services are becoming subscription based, so 5$ here, 5$ there and the cost creeps further.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

I explored building exactly that but turns out there’s no money in it. Most people want the narrative with the facts, if not more so.

Re: Docker to rate limit image pulls

#165

Post by Spivak »

pjmlp wrote: post #132
mm89 wrote: post #119

Earlier quoted context omitted.

Exactly. It's hard enough to save money as it is. News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased…

Mind blowing, imagine paying $5 for each newspaper one wants to read.

You’re missing the point. If there’s a news source that you occasionally read then it’s far more cost effective for you to just buy the paper at the stand for 50¢ the few times you want it. Same with magazines. If you read every newspaper then getting lower cost and delivery in return for the paper getting consistent revenue is a good deal for both parties.

If you get your news like most people, via link aggregators like HN, Reddit Facebook, Twitter then you get linked to dozens of publications that all want a $5/mo. commitment which is untenable.

Re: Docker to rate limit image pulls

#166

Post by Spivak »

heavenlyblue wrote: post #74
nrmitchi wrote: post #51

Earlier quoted context omitted.

It sounds like it definitely does trigger the rate limiter. > There is a small tradeoff – if you pull an image you already have, this is still counted even if you don’t download the layers. I expect we're just going to see a lot more recycling of build nodes once it has "used up it's docker credits".

All reasonable orgs should have had their private docker repo a long time ago. Everybody else is living the pipe dream where they have externalised their risk and probably deserve the Docker treatment.

Yes but Docker achieved their goal of making it annoying as hell to not user DockerHub. Because you can run your own private repo just fine but what you want is a transparent proxy (like apt-catcher) that will let you pretend you’re using DH but actually pulling from either the cache or your private repos. All the pieces are there with private repos and “pullthrough” proxies they’re just not well integrated, seemingly on purpose.

RedHat’s patches to Docker make this possible but Docker has refused to upstream it.

Re: Docker to rate limit image pulls

#167

Post by cpuguy83 »

kinghajj wrote: post #150
cpuguy83 wrote: post #121

Earlier quoted context omitted.

This is not true. There is, effectively, no "virtualization" layer here. There are some things that if needed can cause overhead... such as the bridge networking (really shouldn't be a bottleneck for majority of people), and the CoW filesystem... which docker won't be (or shouldn't be) running on top of since, for example, overlayfs on top of overlayfs is not supported. There is also nothing stripped down about the d…

Sure, I was speaking off the cuff based on my experience from a few years ago. Maybe I messed up and somehow had the DinD daemon not use a volume mount, and that's what caused it to build images slowly?

Very well could be since it would have to fallback to the naive graphdriver that just copies stuff around.

Re: Docker to rate limit image pulls

#168

Post by jacques_chester »

bpfrh wrote: post #87
nickjj wrote: post #54

Does anyone have any good ideas on how the Docker Hub could be monetized in a way that's user friendly and makes sense? AWS, GCP, Azure, DigitalOcean and even GitHub / GitLab all have private container registry offerings. If your stack is on X provider, chances are you're going to use their private registry service instead of using the Docker Hub because you've gone all-in with that provider. That means private repos…

take over supply checks? E.g. make base images for programming languages, systems, etc and guarantee security. I think many small companies would like it a lot better if they got could externalize the cost of running docker images with all their dependencies.

This what Cloud Native Buildpacks do. There are already three major suppliers of these: Heroku, Paketo (by Cloud Foundry folks), and Google Cloud.

I'd be pretty thrilled if Docker encouraged buildpack use. It would be a huge win for them.

Re: Docker to rate limit image pulls

#169

Post by jacques_chester »

Crazyontap wrote: post #19

Honest question, but why can't docker use something like bittorrent to download images? Most of us download our OS via torrents only, so we may as well download the images too if there was support for it.

> Honest question, but why can't docker use something like bittorrent to download images?

Docker will be limiting manifest operations, not actual blob transmissions.

Re: Docker to rate limit image pulls

#170

Post by jacques_chester »

jpalomaki wrote: post #8

I guess CI services like Github Actions could be easily hitting these limits (100 pulls per IP per 6 hours).

I am anecdotally aware of large Concourse installations that have caused individual companies to be blocked entirely from Dockerhub (and which caused "face melting" of Github Enterprise instances).

Badly-behaved CI is a serious issue at scale.

Post reply on HN