Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go Modules Cheat Sheet

encore.dev

Post reply on HN
Print view View original

41–50 of 52 posts

Re: Go Modules Cheat Sheet

#41

Post by morelisp »

gbear0 wrote: post #36
Steltek wrote: post #16

Earlier quoted context omitted.

For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

We set up an internal Athens[0] caching proxy with credentials to pull from our GitLab. So far it seems to be win/win/win

- We don't need this dance in our Dockerfiles / build scripts / dev machines anymore. We have a baked GOPROXY / GONOSUMDB in our build image and developers configure the same proxy (via `go env -w`) locally.

- We pull all packages over the proxy, so builds are faster / use less bandwidth / still mostly work when GitHub goes down.

- SCAs get more difficult; the credentials the proxy has are more limited than any individual developer's GitLab tokens / credentials, and owning the proxy is going to be harder than a single developer's laptop.

[0] https://docs.gomods.io/

Re: Go Modules Cheat Sheet

#42

Post by ramonrue »

gbear0 wrote: post #36
Steltek wrote: post #16

Earlier quoted context omitted.

For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

At least for Gitlab, you can write a .netrc in the user‘s home dir like so:

machine (e.g. gitlab.com) login password

For the access token, you can also leverage Gitlab‘s CI Job Token.

What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI.

(https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html)

Re: Go Modules Cheat Sheet

#44

Post by morelisp »

ramonrue wrote: post #42
gbear0 wrote: post #36

Earlier quoted context omitted.

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

At least for Gitlab, you can write a .netrc in the user‘s home dir like so: machine (e.g. gitlab.com) login password For the access token, you can also leverage Gitlab‘s CI Job Token. What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI. ( https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html )

If you do take this approach, writing your access token in plaintext into a well-known file in your home directory seems like a strictly worse idea than the (also more broadly-compatible) Git insteadof rule to pull via SSH and your local agent.

Re: Go Modules Cheat Sheet

#45

Post by morelisp »

tsimionescu wrote: post #30
gary-kim wrote: post #24

Earlier quoted context omitted.

A change in your repo does not need to result in an import path change if you set up vanity URLs for your import path. For instance, all my go libraries use the import path of gomod.garykim.dev even though the repos are actually hosted on GitHub. That means if I ever change to self-hosting, for example, I just need to update some of the meta tags on the gomod.garykim.dev website to change the repo location.

That still means that if you lose your domain name (gomod.garykim.dev) for whatever reason, you have to modify your code.

You can use a replace directive to handle cases like this without updating the code.

https://golang.org/ref/mod#go-mod-file-replace

Re: Go Modules Cheat Sheet

#46

Post by BigGreenTurtle »

gbear0 wrote: post #36
Steltek wrote: post #16

Earlier quoted context omitted.

For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

What I've done on a recent project is to vendor the dependencies before build(which are excluded from git, you could cache these if you're using a CICD system like circle), copy everything into the build container, build with -mod=vendor and then copy the resulting binary into a new container.

Don't know how that'll work out for a larger project but it works well enough in this small API I wrote.

Re: Go Modules Cheat Sheet

#47

Post by travisd »

gbear0 wrote: post #36
Steltek wrote: post #16

Earlier quoted context omitted.

For a truly private server, you'll need two needles of that mountainous haystack: set GOPRIVATE and tell git to use ssh first: git config --global url.git@HOSTNAME:.insteadOf https://HOSTNAME/

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

For Go, I also take the vendor approach.

But for other stuff (namely private npm packages), (relatively) modern versions of Docker support build-time-only secrets: https://docs.docker.com/develop/develop-images/build_enhance.... Pass the secret as part of the docker build command and then access it inside RUN --mount=... stages.

Re: Go Modules Cheat Sheet

#48

Post by 5dd83f0e »

throwawaygc123 wrote: post #39

i have also written an article given my frustration in finding simple information on the topic ie. how to upgrade go deps[0] which is not immediately easy to find from the extensive Go wiki[1] and a general go mod how-tos tutorial[2] [0] https://golang.cafe/blog/how-to-upgrade-golang-dependencies.... [1] https://github.com/golang/go/wiki/Modules#how-to-upgrade-and... [2] https://www.youtube.com/watch?v=AJxKKmzRTUY

Why did you decide not contributing to the wiki? (curious, not blaming tone)

Re: Go Modules Cheat Sheet

#49

Post by tsimionescu »

morelisp wrote: post #45
tsimionescu wrote: post #30

Earlier quoted context omitted.

That still means that if you lose your domain name (gomod.garykim.dev) for whatever reason, you have to modify your code.

You can use a replace directive to handle cases like this without updating the code. https://golang.org/ref/mod#go-mod-file-replace

Only if you're the final consumer of that module - if you publish it so others use your module, the replace directive will be ignored and they will get a failed build (or have to add their own replace directive for your sub-dependency).

Re: Go Modules Cheat Sheet

#50

Post by morelisp »

tsimionescu wrote: post #49
morelisp wrote: post #45

Earlier quoted context omitted.

You can use a replace directive to handle cases like this without updating the code. https://golang.org/ref/mod#go-mod-file-replace

Only if you're the final consumer of that module - if you publish it so others use your module, the replace directive will be ignored and they will get a failed build (or have to add their own replace directive for your sub-dependency).

Yes, but this is roughly the same as any packaging system - the ones that don't use DNS as a default namespace require you to add explicit package repositories, which is no more or less onerous than the replace directive.
Post reply on HN