Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taking Go modules for a spin

dave.cheney.net

Post reply on HN
Print view View original

41–50 of 75 posts

Re: Taking Go modules for a spin

#41

Post by slrz »

benesch wrote: post #12
bsaul wrote: post #6

I haven’t followed the history of the feature, but could anyone explain why we had to wait until version 11 to be able to build a project from any location we wanted ? I’m having a hard time believing it’s purely for technical reasons, but then i don’t understand either what has changed that makes it a desirable feature now rather than before.

I can't speak to this with absolute certainty, but I have some speculation to offer. The story of GOPATH is tightly intertwined with the story of package management. Go is a Google project, and Google has a very unique approach to package management: commit everything to the monorepo. The GOPATH is, in essence, a monorepo. If you want to change the API of a library, well, you can just change all its callers across yo…

I disagree that GOPATH is only appropriate for monolithic repos. I think it works nicely with distributed open source development and has some nice properties there that are lacking in "project-based" approaches.

It's true that it encourages working with all the code in GOPATH. This is a good thing. Your GOPATH is a view of the whole Go ecosystem. You fix a bug where it makes sense and it is picked up by all users. Sadly, vendoring already messed this up.

I think it's insanity when every program has a different idea of what code a given import path refers to (like is often the case with project-based package managers and vendoring as well). It's no fun to juggle the version differences in your head while working on multiple projects.

Go modules have some good ideas here. Semantic import versioning hopefully reduces the number of different versions you have to consider.

Doing the version selection not per-project but globally for all of GOPATH should still result in a working (but not necessarily reproducible or high-fidelity) build. It definitely reduces the amount of code you need to deal with.

Re: Taking Go modules for a spin

#42

Post by pinkgodzilla »

leadhyena wrote: post #35
pinkgodzilla wrote: post #24

Earlier quoted context omitted.

Can someone help me understand why people hate gopath? We have a script called govars.sh at the root of every project and it sets the gopath exactly the same way one would use virtualenv in Python. We remove the default one from .bashrc or .profile. This basically makes gopath disappear entirely - we never bother explaining to new devs what it "really" means. Just ask them to use our template project folder structure…

In my experience, it's a real pain to get golang projects to build in Jenkins, and it's probably the same for any CI/CD setup, simply because of the gopath. Jenkins agents expect to set up its build directories a very particular way, and so does go with its GOPATH. Juggling it just right to get it to work is frustratingly difficult. In fact, the one main reason I'm looking forward to this release is because of the el…

In gitlab ci, we run govars.sh which not just sets the gopath but also adds the "bin" folder to PATH. Then when we run tests we know where to get our compiled binary from. Our govars also sets the path on where to find our conf.yaml for env config. Didn't know it would cause so many issues in Jenkins!

Re: Taking Go modules for a spin

#43

Post by ainar-g »

dilap wrote: post #29

Huh, vendor/ won't be honored unless you're in GOPATH? Anyone know any more about this? Seems like an odd choice.

See "go help modules".

  Modules and vendoring

  When using modules, the go command completely ignores vendor directories.

  By default, the go command satisfies dependencies by downloading modules
  from their sources and using those downloaded copies (after verification,
  as described in the previous section). To allow interoperation with older
  versions of Go, or to ensure that all files used for a build are stored
  together in a single file tree, 'go mod -vendor' creates a directory named
  vendor in the root directory of the main module and stores there all the
  packages from dependency modules that are needed to support builds and
  tests of packages in the main module.

  To build using the main module's top-level vendor directory to satisfy
  dependencies (disabling use of the usual network sources and local
  caches), use 'go build -getmode=vendor'. Note that only the main module's
  top-level vendor directory is used; vendor directories in other locations
  are still ignored.

Re: Taking Go modules for a spin

#44

Post by dragonwriter »

flurrything wrote: post #38
dilap wrote: post #28

Earlier quoted context omitted.

From cargo's source: "Actually solving a constraint graph is an NP-hard problem. This algorithm is basically a nice heuristic to make sure we get roughly the best answer most of the time." ( https://github.com/rust-lang/cargo/blob/master/src/cargo/cor... ) vgo's more-constrained specification for dependencies means there is exactly one right answer, and it can be easily and quickly calculated by both computer and hum…

Is there a catch to vgo's approach? If not, why aren't the cargo people copying it?

The catch to the vgo approach required that no package in the ecosystem ever have even unintentional backwards incompatibilities, because you can't do anything other than specify minimum versions. Or, rather, it makes the resulting problems something that need to be addressed outside the scope of dependency specification and resolution.

When you just decide not to address a significant part of the problem, the solution becomes simpler.

Re: Taking Go modules for a spin

#45

Post by weberc2 »

majewsky wrote: post #36
weberc2 wrote: post #19

Earlier quoted context omitted.

I like the consistent, sane project structure, but GOPATH is an odd duck to be sure.

I actually started using the GOPATH structure for all my repos after discovering three full checkouts of github.com/torvalds/linux [1] on my drive. [1] I know it's not the official upstream remote, but I find this one the easiest to remember.

I do the same, but I don't like that it is searched by default when building a Go program. It's too easy for the versions of dependencies in those directories to have changed. And even if you vendor, it falls back to GOPATH so you can still bring things into your build by accident.

Re: Taking Go modules for a spin

#46

Post by eikenberry »

dragonwriter wrote: post #44
flurrything wrote: post #38

Earlier quoted context omitted.

Is there a catch to vgo's approach? If not, why aren't the cargo people copying it?

The catch to the vgo approach required that no package in the ecosystem ever have even unintentional backwards incompatibilities, because you can't do anything other than specify minimum versions. Or, rather, it makes the resulting problems something that need to be addressed outside the scope of dependency specification and resolution. When you just decide not to address a significant part of the problem, the soluti…

> unintentional backwards incompatibilities

You mean a bug? Because that's what that is and it is no different from any other bug, and like any other bug they are outside the scope of dependency specifications as they are unintended.

Re: Taking Go modules for a spin

#47

Post by eikenberry »

codezero wrote: post #39
flurrything wrote: post #38

Earlier quoted context omitted.

Is there a catch to vgo's approach? If not, why aren't the cargo people copying it?

I’m no expert, and I might even be very wrong, but I read the post about it and it seems to hinge on only resolving a minimum version and assuming all future packages with the same import path are backwards compatible. If I’m reading right it basically treats path/to/package and path/to/package/v2 as entiresly different packages.

You are correct. It bakes semantic versioning into the dependency system making it a requirement vs. just a convention.

Re: Taking Go modules for a spin

#48

Post by dilap »

ainar-g wrote: post #43
dilap wrote: post #29

Huh, vendor/ won't be honored unless you're in GOPATH? Anyone know any more about this? Seems like an odd choice.

See "go help modules". Modules and vendoring When using modules, the go command completely ignores vendor directories. By default, the go command satisfies dependencies by downloading modules from their sources and using those downloaded copies (after verification, as described in the previous section). To allow interoperation with older versions of Go, or to ensure that all files used for a build are stored together…

Thx.

OK, so that makes it sound like it has nothing to do with GOPATH.

But interesting (annoying?) that vendor won't kick in unless you specifically ask for it.

Re: Taking Go modules for a spin

#49

Post by greenhouse_gas »

dilap wrote: post #28
weberc2 wrote: post #18

Earlier quoted context omitted.

Can you elaborate on why it’s easier to reason about than Cargo et al? I’ve heard a lot of theoretical criticism of Go modules for not taking Cargo’s approach so I’m surprised to hear an experience report to the contrary.

From cargo's source: "Actually solving a constraint graph is an NP-hard problem. This algorithm is basically a nice heuristic to make sure we get roughly the best answer most of the time." ( https://github.com/rust-lang/cargo/blob/master/src/cargo/cor... ) vgo's more-constrained specification for dependencies means there is exactly one right answer, and it can be easily and quickly calculated by both computer and hum…

>Actually solving a constraint graph is an NP-hard problem

How big of a deal is this IRL? Assuming you have 1000 modules, how long should it take to solve the graph?

Re: Taking Go modules for a spin

#50

Post by ainar-g »

dilap wrote: post #48
ainar-g wrote: post #43

Earlier quoted context omitted.

See "go help modules". Modules and vendoring When using modules, the go command completely ignores vendor directories. By default, the go command satisfies dependencies by downloading modules from their sources and using those downloaded copies (after verification, as described in the previous section). To allow interoperation with older versions of Go, or to ensure that all files used for a build are stored together…

Thx. OK, so that makes it sound like it has nothing to do with GOPATH. But interesting (annoying?) that vendor won't kick in unless you specifically ask for it.

So far, I get a feeling that vendoring will be a second-class citizen in the world of modules. Unfortunately, I will not be surprised if the team considers removing the support entirely in a future release. I just hope that a viable alternative will be proposed.
Post reply on HN