Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker to rate limit image pulls

docker.com

Post reply on HN
Print view View original

151–160 of 274 posts

Re: Docker to rate limit image pulls

#151

Post by pbalau »

eblanshey wrote: post #106
pbalau wrote: post #95

Earlier quoted context omitted.

Isn't Docker-in-Docker actually using the host's Docker daemon? I am mounting the docker socket in all my Docker-in-Docker containers, thus all the build tasks running on the same host can share the caches. I guess one could have docker containers that actually run docker, but I don't see a reason to do that...

I was wondering how Docker-in-Docker works, but I couldn't find it dockermented anywhere. If it's using the host's Docker daemon, why do you need to mount the docker socket?

> If it's using the host's Docker daemon, why do you need to mount the docker socket?

There are 2 components for docker: the daemon and the tool used to send commands to the daemon. In order for said tool to be able to send commands to the daemon, it needs a way to communicate with the daemon. Mounting the socket in the container is the easiest method.

I have a "tooling" image that consists of a set of scripts (python code) to do various things ops related. One of the things is to build new images when required. I have a script that given a git commit will detect the images that need to be build and build them. Having my tooling code in a container makes it easier to deploy and use new versions of the tooling code. I don't need anything on the host apart docker itself. No build scripts, no python.

As I said, i could be running the docker daemon inside the container, but that breaks one of my rules related to containers: containers are not virtual machines, they should only run 1 process and the output of that process should be std out.

Re: Docker to rate limit image pulls

#152

Post by viraptor »

nodesocket wrote: post #146
cddotdotslash wrote: post #131

Earlier quoted context omitted.

If your C suite is personally approving a $5/month charge, your organization is likely no where near the size where a change like this from Docker impacts you.

If your C suite requires approving a $5/month charge the company has serious issues. This always bugs me about the HN attitude about spending money, but it's really a extremely frugal developer complex. It's $5 a month, you get insane value out of Docker Hub, just pay it. I come from an entrepreneur attitude, my time is my most precious commodity. I don't optimize minor expenses, I optimize the big picture, my time,…

Yeah, when it's $5/mo blocking good utilisation of multiple $150k people, someone doesn't understand priorities.

Re: Docker to rate limit image pulls

#153

Post by mvanbaak »

mschuster91 wrote: post #136
hamiltont wrote: post #110

Earlier quoted context omitted.

Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid

Just drop a Sonatype Nexus instance on a Docker container somewhere on your network. Alternatively, use Squid if you don't push to the public Docker registry, although you might need to mess around with internal CA for SSL...

Nexus in a container... because storage in containers is such a good idea? Any vps with a disk is probably a better idea

Re: Docker to rate limit image pulls

#154

Post by viraptor »

young_unixer wrote: post #60
Keverw wrote: post #5

I kinda wonder if Docker as a company is struggling. Redhat made Podman which is a compatible replacement, Then there's swarm but apparently that's not recommended and actively developed anymore, then as far as I know they sold off their enterprise clustering product. Seems Kubernetes is the popular thing now even if a bit complex to setup. Wonder what the current business model? Pretty neat idea of using containers,…

> Redhat made Podman which is a compatible replacement I would actually prefer if they made an incompatible replacement. Docker's CLI is pretty bad in my opinion. I want to use Docker the same way I use a headless virtual machine running an SSH server. I want starting/exiting containers to be independent from their 'main process'. I want to attach/detach whenever I need to and execute arbitrary processes. -- Just use…

> I want to attach/detach whenever I need to and execute arbitrary processes.

Isn't "docker exec -ti container-id /arbitrary/command" enough for that?

Re: Docker to rate limit image pulls

#155

Post by drzaiusx11 »

oppositelock wrote: post #125
hamiltont wrote: post #110

Earlier quoted context omitted.

Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid

I run tens of thousands of docker images in production, or rather, tens of thousands of copies of a few hundred images. If you do something like this, you absolutely MUST have a local registry. Harbor [1], JFrog [2], and Quay [3] would be the first ones that I look at. Harbor is open source, free, and a member of the CNCF. You will need to do a little bit of work to set it up to scale properly. JFrog offers a SaaS re…

I was a happy user of JFrog's registries via site license at my last 2 places. Seemed to just work as expected. Didn't have visibility into the cost though (other teams set it up) so I had no idea it was $3k/year.

Re: Docker to rate limit image pulls

#156

Post by 1337shadow »

mvanbaak wrote: post #153
mschuster91 wrote: post #136

Earlier quoted context omitted.

Just drop a Sonatype Nexus instance on a Docker container somewhere on your network. Alternatively, use Squid if you don't push to the public Docker registry, although you might need to mess around with internal CA for SSL...

Nexus in a container... because storage in containers is such a good idea? Any vps with a disk is probably a better idea

You can still bind mount a directory into a container...

Re: Docker to rate limit image pulls

#157

Post by viraptor »

offbynull wrote: post #114

Why don't they move to a P2P model? Something like BitTorrent DHT or IPFS? DockerHub itself can just be used if the image is scarce and to penalize free riders.

Ipfs is already implemented. https://blog.bonner.is/docker-registry-for-ipfs/

But it's on you to use it. (And it doesn't solve the metadata queries the way I understand it)

Re: Docker to rate limit image pulls

#158

Post by trhway »

msla wrote: post #109
rubyn00bie wrote: post #99

I originally came here to ask how folks use that many docker images in six hours (I'm mostly a Docker n00b, and not being facetious); however, after reading the article, I clicked to see how much unlimited is and it's $5 a month. Now my question has changed to: is $5 a month really a deal breaker for folks to get unlimited? Or what are the use cases where the cost is prohibitive? Open source or community projects? In…

$5/month isn't $5/month. It's convincing your boss you need $5/month, because they need to convince their boss, which eventually makes its way up the chain to C-levels, who don't know Docker from yesterday's rotting tuna casserole and view eating either that or the $5/month with the same level of disdain. It isn't about the money, it's about the Mommy-May-I up and down the chain with emails and meetings and careful e…

>It's convincing your boss you need $5/month, because they need to convince their boss, which eventually makes its way up the chain to C-levels, who don't know Docker from yesterday's rotting tuna casserole and view eating either that or the $5/month with the same level of disdain.

this is where miracle of enterprise sales happen - the $5 subscription can be sold as a $50K+ deal by smooth enterprise sales who will provide the C-exec with the experience making him feel like he did something smart and great for the company.

Re: Docker to rate limit image pulls

#159

Post by tinco »

AaronFriel wrote: post #2

This seems incredibly likely to break development use cases at both extremes: CI/CD systems and developers just starting out could end up pulling quite a few images per hour. Imagine if NPM, Ruby Gems, and so on rate limited package downloads until you paid! I'm not sure if there's a better way to monetize the Docker Hub, but this seems so hostile to adoption.

$5 per month for when you pull so many images it's basically abuse is user hostile?

No developer starting out is going to hit 200 images in 6 hours, and even if the did, they would go "heh" and then either take a break or pony up the 5$.

5$! It's way too little money for those limits, there should be brackets all the way up to 5000$ per month. Same for Rubygems and NPM. It's ridiculous that those are struggling organizations that can barely afford to have professionals work on them, when they're absolutely essential to whole industries.

I wish they would force us to pay them.

Re: Docker to rate limit image pulls

#160

Post by mvanbaak »

grumple wrote: post #82
solatic wrote: post #58

Earlier quoted context omitted.

> This is significant - if your organization only uses a few dozen base images from DockerHub, those images will only be downloaded by each build node _once_, then never again. You're assuming that the set of build nodes is relatively static. Plenty of architectures set up autoscaling for the underlying nodes, that terminate servers that aren't being used and relatively soon enough (tens of minutes, hours) spin up ne…

If you’re using docker for production distribution of images, you should be paying for it. That’s exactly the behavior that creates the need for a limit.

Or don’t use docker in production ;)
Post reply on HN