Live data from Hacker News

Go at Digital Ocean

speakerdeck.com

51–60 of 107 posts

Re: Go at Digital Ocean

#51
Can someone explain slides 54-58, about how it takes many minutes to lowercase strings?

slide 55 reads: "each string operation takes 21 seconds" (an amount of time which I translate conservatively into tens of billions of operations or gigabytes of in-memory lookups).

How can performance be that bad - I would think it's trivial? Like, I'm not getting what lowercasing can possibly be doing that is so resource intensive. Any ideas?

Re: Go at Digital Ocean

#52

Earlier quoted context omitted.

Go vendoring gets such a bad rap undeservedly. In practice it's the simplest and most straightforward dependency system (since 1.4/5 added vendoring) I've used. Step 1: Check out code that you want to use. That's actually the only step and you can do it by hand, or with one of a multitude of tools. All the tools differ slightly, and any metadata or manifests or things like that they do aren't compatible, but the code…

The fact that the full url is used to import packages make it impossible to fork a repository. Once you forked it, you have to change all the imports across the repo, and then it's kinda either very hard to make Pull Requests, or pull the new commits from the original repo. This is one thing I really dislike about go dependencies. If you have a solution for this, let me know, I would be very interested to know more a…

You don't have to change the package name just because it's forked.

I threw together an example of doing this with `dep`. I forked Fatih's color package, and am using it as `github.com/fatih/color` no problem: https://github.com/sofuture/colortest (forked dep here: https://github.com/sofuture/color)

Re: Go at Digital Ocean

#53

Earlier quoted context omitted.

Go vendoring gets such a bad rap undeservedly. In practice it's the simplest and most straightforward dependency system (since 1.4/5 added vendoring) I've used. Step 1: Check out code that you want to use. That's actually the only step and you can do it by hand, or with one of a multitude of tools. All the tools differ slightly, and any metadata or manifests or things like that they do aren't compatible, but the code…

The fact that the full url is used to import packages make it impossible to fork a repository. Once you forked it, you have to change all the imports across the repo, and then it's kinda either very hard to make Pull Requests, or pull the new commits from the original repo. This is one thing I really dislike about go dependencies. If you have a solution for this, let me know, I would be very interested to know more a…

You don't have to use the full url to import a package. The name of a git repository doesn't have to match the remote name though, so I don't see why it would be a problem to fork a repository.

Re: Go at Digital Ocean

#54

Earlier quoted context omitted.

The existence of the GOPATH environment variable is not a problem. The fact that the toolchain is violently opinionated about where you locate your working copies, and goes out of its way to fight you if you try to fake it with symlinks, is. The GOPATH issue will be solved when I can clone a project to anywhere in $HOME and build it without issue.

This is how I felt when I started using go, but I was unable to rationalize why beyond 'it's different'. What's the real difference between `cd ~/dev` and `cd $MYGO` (~/dev/go/src/github.com/myusername)?

The difference is that I don't want all my Go programs in a different place. I have my place where I put projects. I have ~/projects, ~/oldprojects, ~/defunctprojects. I have ~/contribprojects for things that aren't mine. I decide where my files go.

Re: Go at Digital Ocean

#55

The go vendoring stuff and GOPATH sounds like a complete nightmare to me. Also the fact that imports are full git URLS sounds totally crazy. Is `dep` fixing all that stuff? Last time I checked out go a few years ago, I didn't continue because I found the story of actually installing dependencies and keeping track of versions very confusing. You couldn't even `go get` a specific git tag or something. I'm happy that th…

Go vendoring gets such a bad rap undeservedly. In practice it's the simplest and most straightforward dependency system (since 1.4/5 added vendoring) I've used. Step 1: Check out code that you want to use. That's actually the only step and you can do it by hand, or with one of a multitude of tools. All the tools differ slightly, and any metadata or manifests or things like that they do aren't compatible, but the code…

I agree venturing is actually good with just a simple folder, though IMO (with the benefit of hindsight), gopath was a mistake, and a per project vendor folder for dependencies would make much more sense for collaborative projects outside large companies.

Go get does have support for tags, but it was intended for they have code in go get to find tags like go1 and go2 which are as yet unused (I wish they'd used it for dependency version tags like v5.2 etc instead, it may never be used):

https://golang.org/src/cmd/go/internal/get/get.go#L524

Really some simple additions to go get would have gone a long way - recognise semantic version tags like v1.2 on go get and put them in vendor with some command like 'go vendor my/dep -v 1.2'.

Not really sure they need diamond dependencies, updating them automatically up to version x, manifest+lock files and all the other intricacies which in theory a package manager should solve - humans can resolve them as they come up and in real life use they're not a huge deal (as the incredibly simple go get we currently have shows).

Re: Go at Digital Ocean

#56

Earlier quoted context omitted.

The fact that the full url is used to import packages make it impossible to fork a repository. Once you forked it, you have to change all the imports across the repo, and then it's kinda either very hard to make Pull Requests, or pull the new commits from the original repo. This is one thing I really dislike about go dependencies. If you have a solution for this, let me know, I would be very interested to know more a…

You don't have to use the full url to import a package. The name of a git repository doesn't have to match the remote name though, so I don't see why it would be a problem to fork a repository.

Let say I would like to fork "echo".

Echo is using some imports to their own packages inside the project.

IE: https://github.com/labstack/echo/blob/a098bcd3b0c445dde3d380...

So if I fork "echo", I have to replace the urls in their imports so it imports my fork packages.

Re: Go at Digital Ocean

#58

Earlier quoted context omitted.

The existence of the GOPATH environment variable is not a problem. The fact that the toolchain is violently opinionated about where you locate your working copies, and goes out of its way to fight you if you try to fake it with symlinks, is. The GOPATH issue will be solved when I can clone a project to anywhere in $HOME and build it without issue.

This is how I felt when I started using go, but I was unable to rationalize why beyond 'it's different'. What's the real difference between `cd ~/dev` and `cd $MYGO` (~/dev/go/src/github.com/myusername)?

Well imagine you use language foo, bar and go. Go insists all code lives in ~/go/src and foo insists it all lives in ~/foo/source. Neither likes symlinks, which one wins?

Also people like to keep other artefacts alongside source code like scripts, resources, templates, tools etc and often have an existing elaborate project structure in order to accommodate that. Go forces them to throw that out and use something under gopath. It's annoying and an unnecessary stumbling block for people starting out in Go.

Re: Go at Digital Ocean

#60

Can someone explain slides 54-58, about how it takes many minutes to lowercase strings? slide 55 reads: "each string operation takes 21 seconds" (an amount of time which I translate conservatively into tens of billions of operations or gigabytes of in-memory lookups). How can performance be that bad - I would think it's trivial? Like, I'm not getting what lowercasing can possibly be doing that is so resource intensiv…

Looks like the output of pprof, so perhaps the slide means all calls to this function (when run 10000 times say during checking dependencies over a very large codebase), take 21 second in total, reduced to 4 seconds by storing the results of comparisons in a map and using that instead. Go strings are immutable so calling ToLower repeateadly would copy them each time.
Post reply on HN