Earlier quoted context omitted.
Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid
Just drop a Sonatype Nexus instance on a Docker container somewhere on your network. Alternatively, use Squid if you don't push to the public Docker registry, although you might need to mess around with internal CA for SSL...
Docker to rate limit image pulls
251–260 of 274 posts
Re: Docker to rate limit image pulls
#252Earlier quoted context omitted.
> the current culture regarding builds and CI in general is horrifyingly ineffficient I used to be in charge of the website for a company you’ve heard of. We once realized some huge proportion of our traffic originated from a hosted CI company requesting the site thousands and thousands of times (guessing one for each build they hosted) every 5 minutes. I can’t remember what proportion of traffic it was but I’m prett…
I sure hope the "cloud first" advocates are happy now, because they have managed to create masses of developers who have next to no idea that what they're doing is an incredible waste of resources. These are also the same people who are perplexed why their systems intermittently fail, or are surprised that they do when the Internet connection cuts out for a bit.
Re: Docker to rate limit image pulls
#253Earlier quoted context omitted.
Just drop a Sonatype Nexus instance on a Docker container somewhere on your network. Alternatively, use Squid if you don't push to the public Docker registry, although you might need to mess around with internal CA for SSL...
Nexus in a container... because storage in containers is such a good idea? Any vps with a disk is probably a better idea
Re: Docker to rate limit image pulls
#254Earlier quoted context omitted.
If you cannot get a TLS cert for internal infrastructure in a few minutes, I'd recommend you start looking into why.
no good document on it and it is not very important for me ( I run it on homelab). still wonder how to do it in minutes.
Expect to spend 1-2 hours first time you try it until you can setup the correct DNS records, API keys and configuration.
Afterwards it's pretty hands off, every three months you'll receive an email from letsencrypt and you'll have to rerun this script to regenerate your certificates. Takes 2-3 minutes max (but of course you still need to distribute your certificates to all relevant services...)
Re: Docker to rate limit image pulls
#255I see that Docker doesn't actually offer an AWS-style enterprise account that one can use to hand authorization to developers without requiring those developers to make individual accounts.
It feels pretty sassy of docker to give everyone 2 months to shove credentials everywhere when docker themselves haven't done the minimum to make enterprise accounts realistic. Instead, they're adopting the github model of "oh, just ask everyone to make personal accounts and then include their personal accounts in the org team". That has problems.
Firstly, it puts employers in the unpleasant position of attempting to compel employees to make legal agreements with third parties (docker, in this case). The correct way to do this is AWS-style, where the org itself makes /one/ agreement and then delegates that agreement via access keys. This is the minimum I expect from enterprise account systems, hard fail for docker.
Secondly, it's a clusterfuck to manage. You end up with an org filled with random-arse account names that you can't really audit, and you don't know who has access to what. If employees leave the org, it's hard to ensure that their access is revoked because the access takes place entirely outside the standard account domains.
Github has recently improved this a shade by adding ADFS authorization to org accounts, but that involves asking employees to tie their personal (and all github and docker accounts /are/ personal) account to their work ADFS account, which is a shitty half-solution.
All things considered, docker made this problem for themselves. They've spent /years/ working hard to get everyone to make docker accounts and push everything to docker hub instead of fostering an ecosystem of registries by different orgs for different purposes. All of a sudden it's now "too expensive" and they're dropping the hammer on everyone to sign up and push credentials everywhere with very little warning, whilst not doing their half of the work by making a proper delegated authority account system.
Doesn't fill me with confidence for their future as a stable platform on which to base a business.
Re: Docker to rate limit image pulls
#256Earlier quoted context omitted.
"Protesters say their peaceful assembly has been disrupted and escalated by the police. The police argue they've only been intervening in riots to maintain order and protect property." Done.
If you have to represent "both sides" (in many cases there'll be more than two sides really), you end up having to give a voice to nutjobs, plus you present both sides as equally valid assessments. Much as we'd all love an "unbiased" news source, the reality is that bias is a very hard problem to solve well.
If one side is lying, objective reporting would tell you which side it was.
Re: Docker to rate limit image pulls
#257Earlier quoted context omitted.
Artifactory is less bad than most of the tools I have to use all day.
Artifactory is the very definition of expensive (even at an enterprise scale) when it comes to docker images though.
Working for the same sized companies for a while has apparently dulled my senses. At a certain size, the capital that matters is the political capital it takes to get a vendor agreement in place to begin with. The monthly costs of the system are something you only feel through pushback on how big the repo gets, or the rate of traffic (experiencing the latter now with a browser testing SaaS)
Re: Docker to rate limit image pulls
#258Earlier quoted context omitted.
Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid
I run tens of thousands of docker images in production, or rather, tens of thousands of copies of a few hundred images. If you do something like this, you absolutely MUST have a local registry. Harbor [1], JFrog [2], and Quay [3] would be the first ones that I look at. Harbor is open source, free, and a member of the CNCF. You will need to do a little bit of work to set it up to scale properly. JFrog offers a SaaS re…
Re: Docker to rate limit image pulls
#259Earlier quoted context omitted.
Do you run a local registry? Any high-quality articles/youtube talks to share? I'm about to set one up for our own little cluster (~5 machines, ~75 containers). I know tons about docker engine, and a fair bit about the registry, but it's always nice to watch a "lessons learned from actually doing this in production" talk to know what mistakes to avoid
I just use ECR[1] which in many cases costs less and is fully locked down behind my AWS VPC With ECR you pay for image storage: $0.09 per GB after the first 1 GB which is free [1] https://aws.amazon.com/ecr/
Re: Docker to rate limit image pulls
#260Good! I'd even welcome much more agressive limits than what they're proposing; the current culture regarding builds and CI in general is horrifyingly ineffficient, wasteful and in the end just plain slow. I'm looking forward to developers adjusting their workflows (and caches, etc.) to actual, reasonable limits, not just perusing the service as if it were an unlimited cost-free cornucopia of software.