Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taking Go modules for a spin

dave.cheney.net

Post reply on HN
Print view View original

21–30 of 75 posts

Re: Taking Go modules for a spin

#21

Post by hellcow »

weberc2 wrote: post #18
hellcow wrote: post #15

I've been using go modules in my company for several months now. Everything has "just worked." It's at least 1 order of magnitude faster than dep. It's worth noting that go modules use a novel dependency resolution algorithm which is extremely simple to reason about/implement, fast, and produces more reliable builds than npm/bundler/cargo. That's why I was excited about it, anyway. It removes the ever-present NP-comp…

Can you elaborate on why it’s easier to reason about than Cargo et al? I’ve heard a lot of theoretical criticism of Go modules for not taking Cargo’s approach so I’m surprised to hear an experience report to the contrary.

I can't do much justice to the topic in a HN post, but this post specifically describes Go module's dependency resolution algorithm and compares it against SAT solvers like Cargo's: https://research.swtch.com/vgo-mvs. Click the "Go & Versioning" link in the header, and you'll find a whole series explaining the design decisions.

Re: Taking Go modules for a spin

#22

Post by benesch »

atombender wrote: post #17
benesch wrote: post #12

Earlier quoted context omitted.

I can't speak to this with absolute certainty, but I have some speculation to offer. The story of GOPATH is tightly intertwined with the story of package management. Go is a Google project, and Google has a very unique approach to package management: commit everything to the monorepo. The GOPATH is, in essence, a monorepo. If you want to change the API of a library, well, you can just change all its callers across yo…

The Go team, for years, actually insisted that package management was something the Go community had to go and figure out. As you say, Google uses a monorepo where they check everything into a single tree. One of the core Go developers -- I forget who, unfortunately -- actually claimed at one point that they didn't want to design a package management system because they didn't know how ; since Google used a monorepo,…

> My personal theory is that what made Russ Cox cave in was his discussions with Sam Boyer. Cox thought Boyer was going down the wrong path, and thought he had a better solution.

That's certainly my understanding of the situation. Matt Farina has a great commented history of dep and vgo [0] if you haven't already seen it. The comments are particularly enlightening.

Still, it's not clear to me what made the Go team get into the package management game at all. As you say, for years they were happy to leave that as a community problem. But something spurred them to declare that Dep was an "official experiment."

> Dep is okay when it works, but inherits pretty much all the warts of Glide, which Boyer also worked on.

Funny, I've had exactly the opposite experience. Glide caused us plenty of trouble at CockroachDB, but Dep has worked flawlessly, if slowly. I've also found Sam to be exceptionally friendly and responsive to feedback [1] [2].

[0]: https://codeengineered.com/blog/2018/golang-godep-to-vgo/#co...

[1]: https://github.com/golang/dep/issues/1927

[2]: https://github.com/golang/dep/issues/460

Re: Taking Go modules for a spin

#23

Post by atombender »

benesch wrote: post #22
atombender wrote: post #17

Earlier quoted context omitted.

The Go team, for years, actually insisted that package management was something the Go community had to go and figure out. As you say, Google uses a monorepo where they check everything into a single tree. One of the core Go developers -- I forget who, unfortunately -- actually claimed at one point that they didn't want to design a package management system because they didn't know how ; since Google used a monorepo,…

> My personal theory is that what made Russ Cox cave in was his discussions with Sam Boyer. Cox thought Boyer was going down the wrong path, and thought he had a better solution. That's certainly my understanding of the situation. Matt Farina has a great commented history of dep and vgo [0] if you haven't already seen it. The comments are particularly enlightening. Still, it's not clear to me what made the Go team ge…

Indeed, as I said, Dep is okay when it works, until it doesn't. This [1], for example, is a blocking issue, and requires some manual editing of the lock file to get around. I've had other issues. The issues pale in comparison to the horribleness of Glide, but it's interesting just how these tools end up being so damn flaky.

[1] https://github.com/golang/dep/issues/1207

Re: Taking Go modules for a spin

#24

Post by pinkgodzilla »

benesch wrote: post #12
bsaul wrote: post #6

I haven’t followed the history of the feature, but could anyone explain why we had to wait until version 11 to be able to build a project from any location we wanted ? I’m having a hard time believing it’s purely for technical reasons, but then i don’t understand either what has changed that makes it a desirable feature now rather than before.

I can't speak to this with absolute certainty, but I have some speculation to offer. The story of GOPATH is tightly intertwined with the story of package management. Go is a Google project, and Google has a very unique approach to package management: commit everything to the monorepo. The GOPATH is, in essence, a monorepo. If you want to change the API of a library, well, you can just change all its callers across yo…

Can someone help me understand why people hate gopath? We have a script called govars.sh at the root of every project and it sets the gopath exactly the same way one would use virtualenv in Python. We remove the default one from .bashrc or .profile. This basically makes gopath disappear entirely - we never bother explaining to new devs what it "really" means. Just ask them to use our template project folder structure.

Re: Taking Go modules for a spin

#25

Post by jen20 »

bsaul wrote: post #11
jakoblorz wrote: post #8

Earlier quoted context omitted.

Go has always been extensively opinionated (gofmt is just one example of that). This is also the beauty of Go - everything is just as expected once you understand the "opinions". The rather late support of "out-of-tree" building might be caused by a conflict between groups pressing to refrain from the $GOPATH approach (rising in size due to rising Go popularity itself) and Go Dev Team / early-adopters especially as t…

I'm having a hard time believing the go core team changed its mind upon popular pressure only. Are you sure there wasn't a real-life painful scenario that appeared or grew in importance , and convinced them an alternative system had to be provided ?

The recent(ish) interview with Russ Cox on the "Go Time" podcast [1] sheds some light on this, but I listened to it long enough ago that I can't remember the entire reasoning.

[1]: https://changelog.com/gotime/77

Re: Taking Go modules for a spin

#26

Post by benesch »

atombender wrote: post #23
benesch wrote: post #22

Earlier quoted context omitted.

> My personal theory is that what made Russ Cox cave in was his discussions with Sam Boyer. Cox thought Boyer was going down the wrong path, and thought he had a better solution. That's certainly my understanding of the situation. Matt Farina has a great commented history of dep and vgo [0] if you haven't already seen it. The comments are particularly enlightening. Still, it's not clear to me what made the Go team ge…

Indeed, as I said, Dep is okay when it works, until it doesn't. This [1], for example, is a blocking issue, and requires some manual editing of the lock file to get around. I've had other issues. The issues pale in comparison to the horribleness of Glide, but it's interesting just how these tools end up being so damn flaky. [1] https://github.com/golang/dep/issues/1207

Ah, that is an unfortunate issue, and the error message is unreadable to boot.

> The issues pale in comparison to the horribleness of Glide, but it's interesting just how these tools end up being so damn flaky.

That's the crux of it, isn't it? The dozens of Go package managers that have come and gone over the years have provided us with substantial evidence that building a stable package manager requires several years of development. I think that's why I'm frustrated that the Go team hit the reset button again. Dep has accumulated plenty of bug fixes over the years to handle more and more of these edge cases, but vgo had to start from scratch.

On the bright side, vgo essentially can't fail.

Re: Taking Go modules for a spin

#27

Post by e12e »

chipaca wrote: post #4

Given it's 18.04, you could've `snap install go --channel=edge` to get go from master. (all praise mwhudson for maintaining the Go snaps -- `snap info go` for the whole story).

I've recently revisited asdf, and based on some testing, started moving my various compilers/interpreters to that. It's a "general" version manager - that works like rbenv for "all" languages.

I'm not sure if I'd use it for deployment - but for development it's quite versatile.

https://github.com/asdf-vm/asdf

Re: Taking Go modules for a spin

#28

Post by dilap »

weberc2 wrote: post #18
hellcow wrote: post #15

I've been using go modules in my company for several months now. Everything has "just worked." It's at least 1 order of magnitude faster than dep. It's worth noting that go modules use a novel dependency resolution algorithm which is extremely simple to reason about/implement, fast, and produces more reliable builds than npm/bundler/cargo. That's why I was excited about it, anyway. It removes the ever-present NP-comp…

Can you elaborate on why it’s easier to reason about than Cargo et al? I’ve heard a lot of theoretical criticism of Go modules for not taking Cargo’s approach so I’m surprised to hear an experience report to the contrary.

From cargo's source: "Actually solving a constraint graph is an NP-hard problem. This algorithm is basically a nice heuristic to make sure we get roughly the best answer most of the time." (https://github.com/rust-lang/cargo/blob/master/src/cargo/cor...)

vgo's more-constrained specification for dependencies means there is exactly one right answer, and it can be easily and quickly calculated by both computer and human.

Whether or not this will turn out to matter in practice is still an open question.

Re: Taking Go modules for a spin

#30

Post by irq-1 »

Go is forever changing the pathing and configuration. Even here we have:

> Very nice, go build ignored the vendor/ folder in this repository (because we’re outside $GOPATH)

and

> Oddly these are stored in $HOME/go/src/mod not the $GOCACHE variable that was added in Go 1.10

Maybe Go 2.0 will be stable.

Post reply on HN