Live data from Hacker News

Running our Docker registry on-prem with Harbor

dev.37signals.com

21–30 of 78 posts

Re: Running our Docker registry on-prem with Harbor

#21
post #17

This looks nice. What would be good on-prem S3 companion for this? I know if minio but I think there was some recent drama about it (I don't know specifics, just a feeling)

If you want to have replication built in, you can give https://garagehq.deuxfleurs.fr/ a try

Thanks, that looks good. Do you have some real experience with this in production?

Re: Running our Docker registry on-prem with Harbor

#22
post #6

I also run Harbor. I use the official Helm chart; it's a little jank, doesn't support a couple of things we want. It only works with one of ArgoCD/ExternalSecretsOperator, and it doesn't support Redis TLS. Contrary to the author of this post, we just run one (the "source of truth") and use caching proxies in other regions. Works fine for us.

What’s jank about it?

I mentioned two things that were broken:

1. Doesn't work with ExternalSecretsOperator and ArgoCD, which I happen to use. This is because the author of the Harbor chart decided not to use k8s concepts like secretRef in a podTemplate. Instead, at Helm template time, it looks up the secret data and writes it into another secret, which is then included as a envFrom. This interacts poorly with ExternalSecretsOperator in general, because it breaks the lifecycle control that ESO has. It's completely broken with ArgoCD because ArgoCD disables secret lookups by charts for pretty valid security concerns. No other chart I've come across does secret lookups during helm template time. Even the helm docs tell you it's not correct.

2. Harbor requires redis, but the Helm chart doesn't correctly pipe in the connection configuration. Redis can't be behind TLS, or the chart won't work.

Re: Running our Docker registry on-prem with Harbor

#23
post #17

This looks nice. What would be good on-prem S3 companion for this? I know if minio but I think there was some recent drama about it (I don't know specifics, just a feeling)

Minio used to be de facto here, but they did a bait and switch recently and removed the UI from the free version. Garage is probably closest to best in class for open source on prem.

Re: Running our Docker registry on-prem with Harbor

#25

harbor is great ... simple to install ... it's the only container registry I ever use

for simple use cases, the official registry is good enough too

If you use paid S3 as the storage layer, then you want to control size.

With the self hosted official registry, the stop-GC-restart process is a PITA.

Re: Running our Docker registry on-prem with Harbor

#27
post #22

Earlier quoted context omitted.

What’s jank about it?

I mentioned two things that were broken: 1. Doesn't work with ExternalSecretsOperator and ArgoCD, which I happen to use. This is because the author of the Harbor chart decided not to use k8s concepts like secretRef in a podTemplate. Instead, at Helm template time, it looks up the secret data and writes it into another secret, which is then included as a envFrom. This interacts poorly with ExternalSecretsOperator in g…

You could always put the helm chart in a Kustomize and change the things you don’t like.

—-enable-helm isn't supported everywhere but Argo definitely allows it

Re: Running our Docker registry on-prem with Harbor

#28

I'm confused on why they decided to populate the cache by replicating the entirety of Docker Hub instead of using a sort of cache that gets populated on the first pull

That was just confusing but it seems like it was 80 repos, all their stuff on Docker Hub not all of Docker Hub.

Re: Running our Docker registry on-prem with Harbor

#29
post #19

Earlier quoted context omitted.

The article mentions Pure Flashblade. Looks like dedicated hardware. https://www.purestorage.com/products/unstructured-data-stora...

Yes, but I'd rather some self hosted software solution

Not S3, but I use longhorn to store persistent data on my clusters.

Sometimes you'll need admin skills, but only if you spread your cluster out over availability zones or poor connections

Re: Running our Docker registry on-prem with Harbor

#30

Why does the Harbor VM need 32 cores and 64GB of RAM? Especially if it's only serving 32,000 pulls over 2 months.

FWIW,

   > During this time, Harbor has served more than 32,000 pulls under company-wide use in day-to-day business.
It is possible to read this as the "32,000 pulls" is a daily number, not a total one.
Post reply on HN