Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running our Docker registry on-prem with Harbor

dev.37signals.com

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Running our Docker registry on-prem with Harbor

#51

Post by tedivm »

lijok wrote: post #35

We self-host Harbor as well, it’s fairly painless. Has SSO out of the box, a Terraform provider that covers everything, and for the most part just works. The issues we’ve had so far: - No programmatic way to retrieve your token that’s required for ‘docker login’. So we had to create a robot account per user and pop their creds into our secrets store. - Migrating between sites by cloning the underlying S3 bucket and s…

The lack of OIDC support for Harbor has been the biggest annoyance for me. I'd love to be able to push from Github Actions to Harbor without needing robot users.

Re: Running our Docker registry on-prem with Harbor

#53

Post by adolph »

01HNNWZ0MV43FF wrote: post #7
fastest963 wrote: post #4

Why does the Harbor VM need 32 cores and 64GB of RAM? Especially if it's only serving 32,000 pulls over 2 months.

I want something like "This could have been an email" but "This could have been a Caddy instance and static files" Hell Git doesn't even need the Git protocol if you do `update-server-info`

I recall mentioned here ttl.sh, which as I looked it up [0], uses through Docker a CNCF project called Distribution Registry [1] which implements the core container registry functions (and appears to have additional utility, like being a pull-through cache).

0. https://github.com/replicatedhq/ttl.sh/blob/main/registry/en...

1. https://distribution.github.io/distribution/

Re: Running our Docker registry on-prem with Harbor

#54

Post by tempest_ »

We run a docker registry on prem as a pull through cache (none of our containers are stored in there) to keep the rate limit reasonable.

It is pretty easy to just run the basic registry for this purpose.

We have a similar setup for NPM and Pypi on the same machine. It doesnt really need a lot of attention. Some upgrades every once and a while.

Re: Running our Docker registry on-prem with Harbor

#55

Post by E39M5S62 »

reactordev wrote: post #43
E39M5S62 wrote: post #14

Earlier quoted context omitted.

It's in Deft's ORD and IAD data centers, using their network for ingress/egress. Still has to go over transit between those two locations.

But that might be baked into their enterprise pricing. Since it’s still “within” Deft. Site to site is common.

It's not within Deft because they rely on transit between ORD and IAD. That was the case a few years ago when I worked there, it's probably still the same.

Re: Running our Docker registry on-prem with Harbor

#56

Post by MPSimmons »

CBLT wrote: post #22
BlackjackCF wrote: post #8

Earlier quoted context omitted.

What’s jank about it?

I mentioned two things that were broken: 1. Doesn't work with ExternalSecretsOperator and ArgoCD, which I happen to use. This is because the author of the Harbor chart decided not to use k8s concepts like secretRef in a podTemplate. Instead, at Helm template time, it looks up the secret data and writes it into another secret, which is then included as a envFrom. This interacts poorly with ExternalSecretsOperator in g…

Is there no Argo plugin for your secret store? In a previous life, we used Argo Vault Plugin to good effect.

Re: Running our Docker registry on-prem with Harbor

#58

Post by yorwba »

delusional wrote: post #37
lijok wrote: post #35

We self-host Harbor as well, it’s fairly painless. Has SSO out of the box, a Terraform provider that covers everything, and for the most part just works. The issues we’ve had so far: - No programmatic way to retrieve your token that’s required for ‘docker login’. So we had to create a robot account per user and pop their creds into our secrets store. - Migrating between sites by cloning the underlying S3 bucket and s…

What the upgrade story like? Their official website makes it sound like a pain (stopping the software, backing up the database, changing the settings syntax, running some installer). I would expect something built for kubernetes to just do the right thing on startup (such that upgrading is simply switching out the image).

I upgraded Harbor before, it was a pain. I think you're encouraged to use their official Helm chart and then it's supposed to be fairly seamless https://goharbor.io/docs/2.13.0/administration/upgrade/helm-... but if your predecessor decided against that option, separately adjusting the configuration for all the moving pieces is fairly annoying. Also, I misconfigured something and ended having to read Harbor source code because the error messages weren't very helpful. Fortunately, I had the presence of mind to first practice on a secondary installation created from a backup. It's definitely not something where you can stop production, install the update, and expect it to come back up in working order.

Re: Running our Docker registry on-prem with Harbor

#59

Post by prmoustache »

nickjj wrote: post #41

> pulling and pushing our images over the internet dozens of times a day caused us to hit the contracted bandwidth limit with our datacenter provider Deft repeatedly I wonder what they were doing that resulted in blowing out their Docker layer cache on every pull and push. Normally only a layer diff would be sent over the wire, such as a code change that didn't change your dependencies.

I'd rather have the agents prune their docker cache (or destroying and recreating agent) every night but it is not uncommon to see pipelines use the --no-cache option at every run to make sure they get the latest security updates.

Re: Running our Docker registry on-prem with Harbor

#60

Post by muhehe »

phireal wrote: post #23
muhehe wrote: post #17

This looks nice. What would be good on-prem S3 companion for this? I know if minio but I think there was some recent drama about it (I don't know specifics, just a feeling)

Minio used to be de facto here, but they did a bait and switch recently and removed the UI from the free version. Garage is probably closest to best in class for open source on prem.

But isn't it still available separately?

https://github.com/minio/object-browser?tab=readme-ov-file

Post reply on HN