Ask HN: How are you using Go to write production-grade back end services? • Hacker News Board

Re: Ask HN: How are you using Go to write production-grade back end services?

#21

Post by closeparen » Fri, Apr 17, 2020, 8:10 AM UTC

Testify for assert and require. GoMock for mocking, but only when it really adds value over a handwritten fake. Table driven tests wherever possible. Don't need heavyweight test frameworks.

Plain HTTP/JSON encouraged for edge-facing services only, typically not in Go. Between services, GRPC, integrated with our metrics, distributed tracing, and auth. When plain HTTP does happen, gorilla/mux.

Wire or FX for DI.

Implicit satisfaction of interfaces is the core of the language. The type system will feel maddeningly obtuse until you learn to use them effectively. It took me a few days to learn the language's structure and start shipping code, but a few years to grok the implications of that simple structure and design software in harmony with it. Interfaces are the key. Think Haskell typeclasses, not Java.

Re: Ask HN: How are you using Go to write production-grade back end services?

#22

Post by WnZ39p0Dgydaz1 » Fri, Apr 17, 2020, 8:59 AM UTC

I've built a pretty complex and high performance go-based microservice architecture (~12 services) as backend with testify and gomock for testing. Pretty happy with those choices, never had any issues. The backend doesn't use HTTP, so no opinion there.

I recently started migrating some of the services to Rust for performance reasons. I would say that Go's biggest strength is perhaps its biggest weakness as well: It "just works". Types are loose (no generics), concurrency is extremely easy. This means I can write working Go code really quickly, but as project complexity grows, code tends to become kind of a mess.

For example, channel-based concurrency can become hard to reason about if you have a complex service. A few times I ended up putting mutexes at various places just to make it work despite knowing that it's not the "right" thing to do. Mutexes then come with their own issues. Once you have a deadlock or race condition, good luck debugging it. There exist multiple packages and tools to detect race conditions and deadlocks, so this seems to be a common problem. I must've spent days or weeks worth of time looking at pprof output. You may say it's my fault as a developer to write sloppy code, and that may be true, but the Go language encourages such code with the decisions it made.

The same goes for types. I never needed generics, but the fact that typing is so loose means you can get away with a lot of sloppy code without being punished for it. This can be great for moving fast, but may come to bite you later on.

Re: Ask HN: How are you using Go to write production-grade back end services?

#23

Post by sethammons » Fri, Apr 17, 2020, 3:18 PM UTC

one2know wrote: post #19

sethammons wrote: post #10

Earlier quoted context omitted.

I've worked in exception based code bases such as Python (Twisted specifically) and Go. Unlike many, I've used both in the same organization working on similar problems for nearly a decade while the scale of number of engineers working in the code and the number of requests it serves have gone up considerably. Hundreds of contributors. Billions of daily requests. I humbly suggest that indexing on the lack of exceptio…

I understand golang was an attempt to fix c's shortcomings and not to create the next c++ or java. But I also understand programmers get their way of doing things that are bad. Case in point, people who do println("entering function doit()"); println ("exiting function doit()"); for every function in the entire codebase because it's how they learned to "debug" and they never tried anything better. Error returns in go…

> As for performant, as of at least ten years ago it became impossible for a person to perceive a performance increase by removing exception handling.

You are really hung up on the exception handling. I never claimed that the error handling made Go more performant. I wrote "performant" as in "there are other things that are important that should be taken into consideration aside from use of exceptions". It might not be as apparent to others now as you edited your original response that was along the lines of "not using exceptions is the worst software decision of all time." (note I don't recall the exact thing you wrote, but that feels close).

As for the new edit you have:

> litered everywhere with if err != nil... garbage

I just grepped one of my Go code bases. It is 40,636 lines of code. It has 326 "if err !=" lines. Less than 1% of the lines are for error handling preamble, and we take our error handling very serious. In fact, a lot of those error checks don't directly bubble up. They perform fallback logic, logging and/or metrics, or set sane default values. Sure, some bubble up, but it is less than half as grep suggests 134 bubble up.

What I value about local error checking is that everything you need is right in front of you as a reader of the code. When I worked in Twisted Python, one particularly bad case of exception handling made it so I literally could not use ErrBacks in one part of the code because in some other module someone used an exception for a non-exceptional error.

Re: Ask HN: How are you using Go to write production-grade back end services?

#24

Post by sethammons » Fri, Apr 17, 2020, 3:30 PM UTC

bradleyankrom wrote: post #13

sethammons wrote: post #7

I've been wanting to do a blog series on how we use Go. Here is a short version. Tests: We use the testing package for unit tests and (maybe too much) use interfaces as arguments so we can create test fakes that behave the way we want so we can validate error paths, logs (yes, we assert on logs), metrics, and, of course, green/good expected behavior. We then have acceptance level testing. These are ensuring the syste…

Appreciate the thoroughness of your comment (if you expand further in blog posts, I’ll read em). How did you settle on your standard project structure and what does it look like?

Here is a fairly typical project structure you would find in one of our projects

    /.buildkite
    /.github
    /acceptance # for housing the acceptance level tests
      /bin # scripts to help
      /config # an anti-pattern, but we like a config package
        /config.go
      /tests
        / # misc go files for testing
      /$foo # any other packages directly related to tests or the environment
    /bin # any scripts to help with anything
    /cmd
     /$appname
        /main.go
      /$bar # any other installable binaries
    /config # an anti-pattern, but we like a config package
      /config.go
    /$raz # packages that support our service
    /internal # stuff we want to import but not let others import. Seldom used.
    /server # or $appname, our main service code
    .gitignore
    .Dockerfile
    README.md
    docker-compose.yml
    go.mod
    go.sum
    makefile

Re: Ask HN: How are you using Go to write production-grade back end services?

#25

Post by hactually » Sat, Apr 18, 2020, 8:14 AM UTC

I've been using standard router and Gorilla Mux for about 5 years now and have so many snippets that I can compose apps with.

I recently built an lru based rate limiter [0] that is compatible with both - it might be useful! Obviously it would need love for multi host but PRs welcome

[0] https://github.com/17twenty/gorillimiter