Go best practices, six years in (2016) • Hacker News Board

Re: Go best practices, six years in (2016)

#51

Post by sdhgaiojfsa » Fri, Mar 02, 2018, 8:22 PM UTC

sagichmal wrote: post #48

sdhgaiojfsa wrote: post #42

Earlier quoted context omitted.

Lots of people have different opinions on this. You're entitled to yours, but I think it's hasty to call it bad advice. I'm not so worried about http.Client behaving incorrectly, but I am worried about the server I'm targeting behaving in a way that's different than my expectations. I'm also worried about getting cookie, post-data, url, etc. formats wrong. Even if they match what I put in my mock, they won't necessar…

> I am worried about the server I'm targeting behaving in a way that's different than my expectations This is the responsibility of system or integration tests, not unit tests. (Unit tests are what's under discussion in the article.) > What harm is done by using the real object, under the circumstances I described? Time, energy, and effort, all of which inevitably lead to longer (slower) dev/test cycles.

> Time, energy, and effort

My contention is that using real objects normally does not take materially more of any of these, and thus that most code is served best by being tested only by integration and regression tests. Where my contention is untrue, apply your best judgment, of course. :)

Re: Go best practices, six years in (2016)

#52

Post by tynorf » Fri, Mar 02, 2018, 9:06 PM UTC

lobster_johnson wrote: post #38

Goopplesoft wrote: post #13

Is anyone else using golang dep[1] as their vendor/dependency tool? I’ve used it and it works great (much better than glide, gb et al) but I don’t see it in the wild too often. [1] https://github.com/golang/dep

We use Dep. It's good, much better than the buggy mess that is/was Glide. My only criticism is that "dep ensure" will actually parse the code to discover dependencies through import statements, which is also what Glide does. To me, this is antithetical to the purpose of a Gopkg.toml/lock file. In other words, Dep's full list of dependencies isn't actually in the Gopkg.toml file; it's a sum of Gopkg.toml and your code…

I'm fairly certain you can use `--flag` with the standard flag package from stdlib: https://godoc.org/flag (in the section on command line flag syntax).

Though supporting `-flag` does remove the very nice combining of short flags.

Re: Go best practices, six years in (2016)

#53

Post by lobster_johnson » Fri, Mar 02, 2018, 9:14 PM UTC

tynorf wrote: post #52

lobster_johnson wrote: post #38

Earlier quoted context omitted.

We use Dep. It's good, much better than the buggy mess that is/was Glide. My only criticism is that "dep ensure" will actually parse the code to discover dependencies through import statements, which is also what Glide does. To me, this is antithetical to the purpose of a Gopkg.toml/lock file. In other words, Dep's full list of dependencies isn't actually in the Gopkg.toml file; it's a sum of Gopkg.toml and your code…

I'm fairly certain you can use `--flag` with the standard flag package from stdlib: https://godoc.org/flag (in the section on command line flag syntax). Though supporting `-flag` does remove the very nice combining of short flags.

Interesting, I didn't know that. I never liked the flag package, and the downside you mention is a reason not to use it. Kingpin (used by the Prometheus projects) and go-flags are nicer. Lots of Google code uses spf13's pflag, which I'm not a fan of, but it does do GNU flags correctly.

Re: Go best practices, six years in (2016)

#54

Post by vorg » Fri, Mar 02, 2018, 9:39 PM UTC

lobster_johnson wrote: post #39

pjmlp wrote: post #14

Earlier quoted context omitted.

Well, it is on its way out, have you missed the news regarding vgo ? https://research.swtch.com/vgo

vgo is at least phrased as being a proposal, not necessarily the future of Go [1]: This post sketches a proposal for doing exactly that, along with a prototype demonstration ... I intend this post to be the start of a productive discussion about what works and what doesn't. Based on that discussion, I will make adjustments to both the proposal and the prototype, and then I will submit an official Go proposal, for int…

The distrust of anything "not invented here" is prevalent in big corps like Google.

Re: Go best practices, six years in (2016)

#55

Post by allover » Fri, Mar 02, 2018, 10:47 PM UTC

kitd wrote: post #18

pjmlp wrote: post #14

Earlier quoted context omitted.

Well, it is on its way out, have you missed the news regarding vgo ? https://research.swtch.com/vgo

Er, thanks. We've just moved to go dep :) Go's dependency management story is starting to resemble JS's modules story :(

> Go's dependency management story is starting to resemble JS's modules story

It's worse tbh. npm was actually always usable if you knew what you were doing. Complaints with npm were generally down to misunderstanding, people not being aware of features like 'npm shrinkwrap' (which has now been replaced with a lockfile-by-default strategy) and of course registry issues (allowing unpublishing) that weren't unique to npm, and were only an issue for people relying on public registries at deploy-time, a bad idea.

Re: Go best practices, six years in (2016)

#56

Post by convolvatron » Fri, Mar 02, 2018, 11:17 PM UTC

sdhgaiojfsa wrote: post #42

sagichmal wrote: post #30

Earlier quoted context omitted.

This is straight-up bad advice. You don't test the http.Client as part of your business logic, even (especially) if your business logic makes HTTP requests. Instead, you trust that package http is already well-tested, mock out http.Client as an abstract HTTP Request Do-er, and you downscope your tests to your business logic exclusively.

Lots of people have different opinions on this. You're entitled to yours, but I think it's hasty to call it bad advice. I'm not so worried about http.Client behaving incorrectly, but I am worried about the server I'm targeting behaving in a way that's different than my expectations. I'm also worried about getting cookie, post-data, url, etc. formats wrong. Even if they match what I put in my mock, they won't necessar…

i wish i could give you 1 thousand points. isolated unit tests can have value in that the bugs they expose are more easily tracked down.

full-system testing, with an honest attempt to do fault injection, by definition exposes all the issues you might have in production. if you have a decent test suite you can honestly claim you can expect it to take client code correctly.

i don't know how thats true if you've only tested A against some stubbed out version of B that always returns success.

integrated testing is harder and requires more framework. its also kinda necessary, and if you only have the resources to do one or the other, the choice seems pretty clear.

i also heartily agree with your comment that since we're supposed to have frictionless and scalable deployment. there is no reason not to test this as part of integration testing and also exploit this function to make integration testing easier.

Re: Go best practices, six years in (2016)

#57

Post by tigershark » Fri, Mar 02, 2018, 11:51 PM UTC

sdhgaiojfsa wrote: post #42

sagichmal wrote: post #30

Earlier quoted context omitted.

This is straight-up bad advice. You don't test the http.Client as part of your business logic, even (especially) if your business logic makes HTTP requests. Instead, you trust that package http is already well-tested, mock out http.Client as an abstract HTTP Request Do-er, and you downscope your tests to your business logic exclusively.

Lots of people have different opinions on this. You're entitled to yours, but I think it's hasty to call it bad advice. I'm not so worried about http.Client behaving incorrectly, but I am worried about the server I'm targeting behaving in a way that's different than my expectations. I'm also worried about getting cookie, post-data, url, etc. formats wrong. Even if they match what I put in my mock, they won't necessar…

It seems a bad advice also in my eyes. I guess that you have never worked on big systems if for you it is normal to bring up all the production dependencies for an integration test. For small projects it may make sense, although I would still prefer to write unit tests to catch exactly were the bug happens and to be independent from breakages in the external systems. The correct approach is to rely heavily on unit tests that run continuously at every build and run the full integration testing suite at regular intervals or overnight. In a quite big project on which I worked many years ago the full integration test suite took 11-12 hours to run, I would be curious how you would proceed in that case given that from what you write it seems that unit tests are pretty useless to you..

Re: Go best practices, six years in (2016)

#58

Post by lobster_johnson » Sat, Mar 03, 2018, 1:38 AM UTC

kylequest wrote: post #33

bbatha wrote: post #29

This is generally great advice, and aside from the package manager recommendations is still relevant today. I do take issue with a few things though. 1. Don't put your internal libraries in /pkg. /pkg has special GOPATH meaning as "compilation cache". Its not an actual name conflict, but why bother risking it. 2. This is just wrong: > fmt.Printf is self-contained and doesn’t affect or depend on global state; in funct…

> 1. Don't put your internal libraries in /pkg. /pkg has special GOPATH meaning as "compilation cache". Its not an actual name conflict, but why bother risking it. It's a little too late for this because it's pretty much an unofficial standard for library code :-) Just look at Kubernetes, Docker or many other major Go-based applications/tools.

Last I checked, pkg is going away as a compilation cache. As of Go 1.10, the compilation is in fact now stored elsewhere: https://golang.org/doc/go1.10

Re: Go best practices, six years in (2016)

#59

Post by sdhgaiojfsa » Sat, Mar 03, 2018, 4:33 AM UTC

tigershark wrote: post #57

sdhgaiojfsa wrote: post #42

Earlier quoted context omitted.

Lots of people have different opinions on this. You're entitled to yours, but I think it's hasty to call it bad advice. I'm not so worried about http.Client behaving incorrectly, but I am worried about the server I'm targeting behaving in a way that's different than my expectations. I'm also worried about getting cookie, post-data, url, etc. formats wrong. Even if they match what I put in my mock, they won't necessar…

It seems a bad advice also in my eyes. I guess that you have never worked on big systems if for you it is normal to bring up all the production dependencies for an integration test. For small projects it may make sense, although I would still prefer to write unit tests to catch exactly were the bug happens and to be independent from breakages in the external systems. The correct approach is to rely heavily on unit te…

> I guess that you have never worked on big systems

I work on search and indexing systems at Google, and have done for more than a decade. I'm by no means the best, nor an authority. But I have spent some time living in this problem space, and this is my conclusion from my years of experience.

FWIW a single run of the tests for the system I work on now takes . . . I don't know. I would guess hours or maybe even a day of computer time? But, computers are cheap, and defects are expensive. So you split the test suite into bits small enough that the whole thing runs in ten minutes, and run them all in parallel. And, yes, there have been cases where I have had to debug the slow startup of a dependency so that I can make it fast enough to run in my test. But, that's what they pay me for, I enjoy that type of work, and it has ancillary benefits to the dependency when it needs to start up in production.

Re: Go best practices, six years in (2016)

#60

Post by tigershark » Sat, Mar 03, 2018, 6:12 AM UTC

sdhgaiojfsa wrote: post #59

tigershark wrote: post #57

Earlier quoted context omitted.

It seems a bad advice also in my eyes. I guess that you have never worked on big systems if for you it is normal to bring up all the production dependencies for an integration test. For small projects it may make sense, although I would still prefer to write unit tests to catch exactly were the bug happens and to be independent from breakages in the external systems. The correct approach is to rely heavily on unit te…

> I guess that you have never worked on big systems I work on search and indexing systems at Google, and have done for more than a decade. I'm by no means the best, nor an authority. But I have spent some time living in this problem space, and this is my conclusion from my years of experience. FWIW a single run of the tests for the system I work on now takes . . . I don't know. I would guess hours or maybe even a day…

In the real world you don’t have pretty much unlimited resources. If I remember correctly those tests I was speaking about were already running in parallel on 2 or 3 machines otherwise the total time would have been 2 or 3 times more. Obviously we couldn’t use hundreds of machines to make them run in 10 minutes. In most of the places were I worked even getting a couple of new machines for production use would take months, I think if I requested even a couple of machines dedicated only for the integration testing I would have been laughed in the face. So if you can run a full suite of integration tests in minutes on hundreds/thousands of machines then good for you, but don’t expect that every company has the resources for doing it. And in those cases unit tests that run continuously are more useful than integration tests that run once a day.