Live data from Hacker News

Docker to rate limit image pulls

docker.com

141–150 of 274 posts

Re: Docker to rate limit image pulls

#141

Earlier quoted context omitted.

If you're doing this, you're in a cloud environment that's also proximate to a blob store, and can trivially host your own registry.

> and can trivially host your own registry That is far from trivial.

And now you have an alternative option - pay $5/month

Re: Docker to rate limit image pulls

#142
Yes, good. After reading the comments on their image retention limits [0] talking about simply pulling the images all the time to keep them fresh, this seems like a reasonable response.

I'll repeat what I wrote there [1]:

If people really think this is a problem, they'd contribute a non-abusive solution. Writing cron jobs to pull periodically in order to artificially reset the timer is abusive.

Non-abusive solutions include:

- extending docker to introduce reproducible image builds

- extending docker push and pull to allow discovery from different sources that use different protocols like IPFS, TahoeLAFS, or filesharing hosts

I'm sure you can come up with more solutions that don't abuse the goodwill of people.

-----------------------------

Additionally, hosting a local network docker repo would mitigate this rate limit completely. Or straight up pay. It's not that difficult. Getting mad about a free, open-source service becoming pay to use... I couldn't imagine the gall and conceitedness.

0: https://news.ycombinator.com/item?id=24143588

1: https://news.ycombinator.com/item?id=24144475

Re: Docker to rate limit image pulls

#143

Earlier quoted context omitted.

Docker-in-Docker (DinD) doesn't piggy back on the host's Docker daemon, but instead runs a stripped-down Docker daemon inside of the container. The major downside is that I/O is quite slow, since you're going through two virtualization layers (the DinD one, plus the host Docker daemon).

Will mounting the socket, as the person I replied to suggested, make it use the host's docker daemon?

Yes, that's the point.

Re: Docker to rate limit image pulls

#144
post #109

Earlier quoted context omitted.

$5/month isn't $5/month. It's convincing your boss you need $5/month, because they need to convince their boss, which eventually makes its way up the chain to C-levels, who don't know Docker from yesterday's rotting tuna casserole and view eating either that or the $5/month with the same level of disdain. It isn't about the money, it's about the Mommy-May-I up and down the chain with emails and meetings and careful e…

If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

> If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

Or has toxic micromanaged structure. I've had friends who have worked places that would barf over ongoing $60/year software charges, where anything like that would have to go up to C levels and require justification. Luckily never worked at one myself, dodged that particular bullet.

Re: Docker to rate limit image pulls

#145

Earlier quoted context omitted.

I can usually max out my connection speed with torrents. This is rarely the case with direct downloads.

The start up time of a torrent is typically so long though, by the time it has connected to all peers and is downloading I have already downloaded the iso with a direct download anyway.

There's a feature of linx-server that provides uploaded files with a torrent URL as well as a regular download URL. I believe the torrent client tries fetching data from peers as well as from linx (http).

https://github.com/andreimarcu/linx-server

Re: Docker to rate limit image pulls

#146
post #109

Earlier quoted context omitted.

$5/month isn't $5/month. It's convincing your boss you need $5/month, because they need to convince their boss, which eventually makes its way up the chain to C-levels, who don't know Docker from yesterday's rotting tuna casserole and view eating either that or the $5/month with the same level of disdain. It isn't about the money, it's about the Mommy-May-I up and down the chain with emails and meetings and careful e…

If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

If your C suite requires approving a $5/month charge the company has serious issues. This always bugs me about the HN attitude about spending money, but it's really a extremely frugal developer complex. It's $5 a month, you get insane value out of Docker Hub, just pay it. I come from an entrepreneur attitude, my time is my most precious commodity. I don't optimize minor expenses, I optimize the big picture, my time, outcomes, and revenue coming in.

Re: Docker to rate limit image pulls

#147

Earlier quoted context omitted.

If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

> If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you. Or has toxic micromanaged structure. I've had friends who have worked places that would barf over ongoing $60/year software charges, where anything like that would have to go up to C levels and require justification. Luckily never worked at one myself, dodge…

My purchase-approval flow is the same for $1 purchases as it is for $1k purchases. If we hadn't finagled a minor workaround, it would be the same as that required for $5k purchases.

Handling each purchase and documenting it in case we are ever audited requires easily $25-50 of people's time.

Re: Docker to rate limit image pulls

#148
post #59

Earlier quoted context omitted.

Sure, now they have to build a proxy...

Super easy to run a docker cache proxy: docker run -d -p 6000:5000 \ -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \ --restart always \ --name registry registry:2 That's it. Now fetch docker images from the IP that command is running on. Taken from gitlab: https://docs.gitlab.com/runner/install/registry_and_cache_se...

I think this addition to docker/config.json should do the trick to make it hit the proxy?

https://docs.docker.com/registry/recipes/mirror/#configure-t...

Re: Docker to rate limit image pulls

#149

Earlier quoted context omitted.

It’s not common to need that many pulls nor is it hard to build your own images. If you’re deploying to a cluster with 200 machines, you could easily hit this if you use the public registry though. However, if you’re managing that size cluster you can probably afford the fee, but more importantly, you should probably pull once to a local registry and use that to deploy to your cluster anyway.

Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid

I just use ECR[1] which in many cases costs less and is fully locked down behind my AWS VPC

With ECR you pay for image storage: $0.09 per GB after the first 1 GB which is free

[1] https://aws.amazon.com/ecr/

Re: Docker to rate limit image pulls

#150

Earlier quoted context omitted.

Docker-in-Docker (DinD) doesn't piggy back on the host's Docker daemon, but instead runs a stripped-down Docker daemon inside of the container. The major downside is that I/O is quite slow, since you're going through two virtualization layers (the DinD one, plus the host Docker daemon).

This is not true. There is, effectively, no "virtualization" layer here. There are some things that if needed can cause overhead... such as the bridge networking (really shouldn't be a bottleneck for majority of people), and the CoW filesystem... which docker won't be (or shouldn't be) running on top of since, for example, overlayfs on top of overlayfs is not supported. There is also nothing stripped down about the d…

Sure, I was speaking off the cuff based on my experience from a few years ago. Maybe I messed up and somehow had the DinD daemon not use a volume mount, and that's what caused it to build images slowly?
Post reply on HN