harbor is great ... simple to install ... it's the only container registry I ever use
And in my experience is the only one that has RBAC and can be deployed on premise and that actually works, I’ve tried everything at this point
? GitLab?
Running our Docker registry on-prem with Harbor
31–40 of 78 posts
Re: Running our Docker registry on-prem with Harbor
#32Earlier quoted context omitted.
I mentioned two things that were broken: 1. Doesn't work with ExternalSecretsOperator and ArgoCD, which I happen to use. This is because the author of the Harbor chart decided not to use k8s concepts like secretRef in a podTemplate. Instead, at Helm template time, it looks up the secret data and writes it into another secret, which is then included as a envFrom. This interacts poorly with ExternalSecretsOperator in g…
You could always put the helm chart in a Kustomize and change the things you don’t like. —-enable-helm isn't supported everywhere but Argo definitely allows it
We just went through this whole Kustomize shenanigan in our company. Seems completely asinine. Why not just fork the chart, fix it yourself?
Re: Running our Docker registry on-prem with Harbor
#33Why does the Harbor VM need 32 cores and 64GB of RAM? Especially if it's only serving 32,000 pulls over 2 months.
I want something like "This could have been an email" but "This could have been a Caddy instance and static files" Hell Git doesn't even need the Git protocol if you do `update-server-info`
It's certainly possible with s3, so probably for any other static host https://news.ycombinator.com/item?id=40942732
Re: Running our Docker registry on-prem with Harbor
#34This looks nice. What would be good on-prem S3 companion for this? I know if minio but I think there was some recent drama about it (I don't know specifics, just a feeling)
https://github.com/seaweedfs/seaweedfs
much simpler than minio
Re: Running our Docker registry on-prem with Harbor
#35We self-host Harbor as well, it’s fairly painless. Has SSO out of the box, a Terraform provider that covers everything, and for the most part just works.
The issues we’ve had so far:
- No programmatic way to retrieve your token that’s required for ‘docker login’. So we had to create a robot account per user and pop their creds into our secrets store.
- Migrating between sites by cloning the underlying S3 bucket and spinning up the new Harbor instance on top of it, does not work. Weird issues with dropping pulls.
- RBAC goes down to project, not repository level, complicating some of our SDLC controls.
- CSRF errors every time you try to do anything in the UI
- Lenient API and lack of docs means things like setting up tag immutability rules via Terraform was a bit of a PITA to figure out the right syntax
So some small issues, but definitely a great piece of software.
Re: Running our Docker registry on-prem with Harbor
#36harbor is great ... simple to install ... it's the only container registry I ever use
And in my experience is the only one that has RBAC and can be deployed on premise and that actually works, I’ve tried everything at this point
I am currently looking into zot, what were your blockers/hiccups with it?
Re: Running our Docker registry on-prem with Harbor
#37We self-host Harbor as well, it’s fairly painless. Has SSO out of the box, a Terraform provider that covers everything, and for the most part just works. The issues we’ve had so far: - No programmatic way to retrieve your token that’s required for ‘docker login’. So we had to create a robot account per user and pop their creds into our secrets store. - Migrating between sites by cloning the underlying S3 bucket and s…
What the upgrade story like? Their official website makes it sound like a pain (stopping the software, backing up the database, changing the settings syntax, running some installer). I would expect something built for kubernetes to just do the right thing on startup (such that upgrading is simply switching out the image).
Re: Running our Docker registry on-prem with Harbor
#38Earlier quoted context omitted.
And in my experience is the only one that has RBAC and can be deployed on premise and that actually works, I’ve tried everything at this point
I am currently looking into zot, what were your blockers/hiccups with it?
When we looked at modernizing our image hosting, it came down to Zot vs Harbor, and we preferred Zot as it looked easier to deploy. Just a go binary with a few environment variables connecting to our minio, what could be easier?
However, when getting the config prod-ready, we started to trip over one thing after the other. First, my colleague was struggling to get the scale-out clustering to work in our container management. Right, use the other deployment way for HA. Then we found that apparently, if you enable OIDC, all other authentication methods get deactivated, so suddenly container hosts would have to login with tokens... somehow? And better hope your OIDC provider never goes down. And then we found a bug on top that Zot possibly doesn't remove blobs from minio during GC.
At that point we reconsidered and went with Harbor.
Re: Running our Docker registry on-prem with Harbor
#39Earlier quoted context omitted.
You could always put the helm chart in a Kustomize and change the things you don’t like. —-enable-helm isn't supported everywhere but Argo definitely allows it
We just went through this whole Kustomize shenanigan in our company. Seems completely asinine. Why not just fork the chart, fix it yourself?
... or the quite common case, make helm write the template once, fix, port to your own process, delete helm, live happy
Re: Running our Docker registry on-prem with Harbor
#40Why does the Harbor VM need 32 cores and 64GB of RAM? Especially if it's only serving 32,000 pulls over 2 months.
FWIW, > During this time, Harbor has served more than 32,000 pulls under company-wide use in day-to-day business. It is possible to read this as the "32,000 pulls" is a daily number, not a total one.
Even if it is a daily number, it's not very much. There are 86,400 seconds in a day. Even limiting the time to an 8 hour business day, this is only around 1 pull per second.