How I write HTTP services in Go after 13 years • Hacker News Board

Re: How I write HTTP services in Go after 13 years

#221

Post by Philip-J-Fry » Sat, Feb 10, 2024, 6:57 PM UTC

jupp0r wrote: post #213

Philip-J-Fry wrote: post #178

Earlier quoted context omitted.

Your dependency structure should just be a tree. It should be inserted literally right next to it's first use case. Your IDE will literally point it to you with red squigglys because the places where you've added a dependency will be missing a parameter. Go to the highest one and add it on the line above.

I've never seen a tree graph, not without lots of global mutable state to cheat around DI. Your logger is just going to be needed almost everywhere. What do you do on shutdown? In languages with destructors, that can automatically give you a call order in reverse of the construction order, but in Go you end up manually ordering things or just not having panicless shutdowns.

Okay, it's not a tree. Because multiple objects will depend on something like a logger. But it's an acyclic graph if designed properly. Which is incredibly simple to setup and teardown.

If your loggers are needed everywhere, then you just pass them as a constructor to the objects that need them. You're literally doing this with fx anyway.

Like, a logger is probably the first thing you new up in main(). So now you can pass it down as a dependency in constructors.

For shutdown you just defer your shutdown functions. Have a basic interface where your services have a Shutdown() method and then you can push them onto a stack and pop them off during shutdown.

There's no manual ordering involved. Your initialisation is a linear top down process, your shutdown is bottom up. It can't be any simpler. If you keep code as close to usage sites then there's only 1 possible order.

Re: How I write HTTP services in Go after 13 years

#222

Post by lelandbatey » Sat, Feb 10, 2024, 7:23 PM UTC

topicseed wrote: post #174

lelandbatey wrote: post #159

Earlier quoted context omitted.

This is kinda missing the point; each handler needs a lot of deps to do it's job, and the most obvious place to put them is in the parameters of the function. That is what I want. I do not want more indirection for aesthetics; I want clarity, even if it's brutal clarity. Whether all the deps are in the method receiver (the parent struct) or in a struct that's a param; it's all just more indirection to hide all the "s…

You do have to instantiate that struct, and you can do it with.... a beautiful NewCreateUser(dep1, dep2, dep3, ..., dep20) *CreateUser {...}. This is essentially what he recommends with his "func newMiddleware() func(h http.Handler) http.Handler".

I'm pointing out that this is basically "passing all the deps at once" with extra steps but no functional benefit; they are at best aesthetic, at worst confusing.

I'd like a world that sacrifices a bit of aesthetics in order to erase ambiguity or confusion. So instead of putting your deps in a struct that's a param, or putting your deps in a parent closure, I'd like to put them in the function params.

Though I will admit that if I had to choose, I'd use (and have used) the closure approach most often.

Re: How I write HTTP services in Go after 13 years

#223

Post by oorza » Sat, Feb 10, 2024, 7:46 PM UTC

HeavyStorm wrote: post #165

oorza wrote: post #56

Earlier quoted context omitted.

That's why I like to tell people to always remember to stay MOIST - the Most Optimal is Implicitly the Simplest Thing. When you add complexity to DRY out your code, you're adding a readability regression. DRY matters in very few context beyond readability, and simplicity and low cognitive load need to be in charge. Everything else you do code-style wise should be in service of those two things.

DRY has nothing to do with readability. The fact that it might help with it is purely coincidental. DRY is about maintainability - if you repeat rules (behavior) around the system and someone comes along and changes it, how can you be sure it affected all the system coherently? I've seen this in practice: we get a demand from the PO, a more recent hire goes to make the change, the use case of interest to the PO gets…

Maintainability and readability are two sides to the same coin. It's not exactly rocket science to cook up an example situation where making a change in one place is less maintainable than making it in two, because of overly DRY, overly abstracted nonsense leading to a _single_ place to change that's so far removed from where you'd expect it to be that it takes much longer and is much more wrought with risk than just having to do it twice.

Doing something twice is not an anathema, that's my point, not when doing it twice is a cognitively easier and practically faster task.

In almost every case, bugs are the result of human error, and keeping cognitive load as low as possible reduces the likelihood of human error in all cases. As DRY as possible is very rarely the lowest cognitive load possible.

Re: How I write HTTP services in Go after 13 years

#224

Post by ndriscoll » Sat, Feb 10, 2024, 8:02 PM UTC

tuyguntn wrote: post #220

ndriscoll wrote: post #219

Earlier quoted context omitted.

Why would you need to update code that isn't matching on the value? It just knows it has an X and passes it to a function that needs an X.

if you don't update the code in intermediate layers, some automated validation based on enum values will fail, which also drops the request

You only need to update the parser and the places that are using it. Depending on language, the parser might update itself (Scala generally works this way). Everyone else has an already parsed value that they're just passing around. That's the point: only run your validation at the outer layer of your application.

Re: How I write HTTP services in Go after 13 years

#225

Post by kaba0 » Sat, Feb 10, 2024, 8:17 PM UTC

zeroCalories wrote: post #155

fbdab103 wrote: post #150

Earlier quoted context omitted.

Definitely use to fall for primitive obsession. It seemed so silly to wrap objects in an intermediary type. After playing with Rust, I changed my tune. The type system just forces you into the correct path, that a lot of code became boring because you no longer had to second guess what-if scenarios.

Yeah, modern type systems are game changers. I've soured on Rust, but if Go had the full Ocaml type system with match statements I think it would be the perfect language.

Go would need such a revamp to be anywhere close to a decent language, that it would be just a straight up other language.

Re: How I write HTTP services in Go after 13 years

#226

Post by jupp0r » Sat, Feb 10, 2024, 8:40 PM UTC

Philip-J-Fry wrote: post #221

jupp0r wrote: post #213

Earlier quoted context omitted.

I've never seen a tree graph, not without lots of global mutable state to cheat around DI. Your logger is just going to be needed almost everywhere. What do you do on shutdown? In languages with destructors, that can automatically give you a call order in reverse of the construction order, but in Go you end up manually ordering things or just not having panicless shutdowns.

Okay, it's not a tree. Because multiple objects will depend on something like a logger. But it's an acyclic graph if designed properly. Which is incredibly simple to setup and teardown. If your loggers are needed everywhere, then you just pass them as a constructor to the objects that need them. You're literally doing this with fx anyway. Like, a logger is probably the first thing you new up in main(). So now you can…

I agree with you on all of this. fx is not doing much more for shutdown than what you describe (calling a handler pushed to a stack created during initialization). Instead of implementing this for every app, I just prefer to use a library with great documentation and tests.

Re: How I write HTTP services in Go after 13 years

#227

Post by xyzzy123 » Sat, Feb 10, 2024, 11:03 PM UTC

jopsen wrote: post #192

xyzzy123 wrote: post #97

Earlier quoted context omitted.

Its a bit icky but LLMs / copilot can speed up the creation of openapi specs a lot. Agree it doesn't fix the "root" problem that the overall syntax is not ergonomic.

My point was that writing an openapi, or other IDL is faster than writing the code to manually do these things. And more accurate than LLMs. Feels like whenever an LLMs could code it, you'd be better of not having the boilerplate code at all.

Agree 100% with all points. I love contract-first. Better for producers and provides multiple tooling options and better docs for consumers.

I was agreeing with parent that some spec formats frankly suck to write by hand and openapi yaml is IMHO one of those (as opposed to say .protos which are nice for humans to read/write).

I use LLM as glorified interactive autocomplete to speed up writing the yaml specs (not the code! Use deterministic generators for that!) and it works great for me, personally (n=1 anecdote).

Re: How I write HTTP services in Go after 13 years

#228

Post by dilyevsky » Sat, Feb 10, 2024, 11:17 PM UTC

flowardnut wrote: post #209

dilyevsky wrote: post #66

Earlier quoted context omitted.

The problem with this approach is writing openapi by hand from scratch is incredibly tedious process. Writing Protobufs, capnproto or any such similar idl feels much more productive

it lacks flexibility but i really enjoy grpc-gateway for 99% of my work https://github.com/grpc-ecosystem/grpc-gateway

gRPC-GW is the way… if you’re fully bought in into Go. A bit harder sell if you’re on other platforms - you need specialized proxy infra

Re: How I write HTTP services in Go after 13 years

#229

Post by MrDarcy » Sun, Feb 11, 2024, 1:37 AM UTC

zemo wrote: post #172

MrDarcy wrote: post #71

Earlier quoted context omitted.

My favorite way to prevent this is to make the config truly immutable, but still configurable with something like this: package config type options struct { name string } type Option func(o *options) func Name(name string) Option { return func(o *options) { o.name = name } } type Config struct { opts *options } func New(opts ...Option) *Config { o := &options{} for _, option := range opts { option(o) } return &Config…

I used that pattern for a while but stopped using it. I first encountered it from this blog post: https://commandcenter.blogspot.com/2014/01/self-referential-... It's a lot of boilerplate to create something that's not actually immutable. It also makes it harder to figure out which options are available, since now you can't just look at the documentation of the type, you have to look at the whole module package to fi…

> It's a lot of boilerplate to create something that's not actually immutable

How is not actually immutable? How could cfg.opts.name be modified after New() returns?

> It also makes it harder to figure out which options are available

I find it easier, the go tooling helps a lot. For example, all the options are grouped together at https://pkg.go.dev/github.com/go-kit/log/level#Option

It's also easy to use "Find Usages" in my editor, and filter on return type.

Re: How I write HTTP services in Go after 13 years

#230

Post by quickthrower2 » Sun, Feb 11, 2024, 1:45 AM UTC

baq wrote: post #217

quickthrower2 wrote: post #184

Earlier quoted context omitted.

I agree on the one hand but empirically I don’t think I have seen a bug where the problem was the string for X ended up being used as Y. Probably because the variable/field names do enough heavy lifting. But if your language makes it easy to wrap I say why not. It might aid readability and maybe avoid a bug. I would probably type to the level of Url, Email, Name but not PersonProfileTwitterLink.

I’ve refactored a large js code base into ts. Found one such bug for every ~2kloc. The obvious ones are found quickly in untyped code, the problem is in rare cases where you e.g. check truthiness on something that ends up always true.

Of those bugs I wondered how much a type would help. For example is it a misunderstanding of business requirements (nosurcharge bool = iscash bool) or a “typo” / copy paste error. If the former types don’t help. The latter they might.