Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

REST Servers in Go: Part 1 – standard library

eli.thegreenplace.net

Post reply on HN
Print view View original

141–149 of 149 posts

Re: REST Servers in Go: Part 1 – standard library

#141

Post by ammanley »

anderspitman wrote: post #114
0xbkt wrote: post #96

I would probably go with gRPC + grpc-gateway[1] instead. Declaring your services and models in proto files, annotating your services with google.api.http to help grpc-gateway scaffold your HTTP base. Then just implement your services from the interface generated by grpc-go. You can even register your gRPC services to grpc-gateway without actually bringing up a gRPC server. You finally end up having your exact data mo…

These things can be fun to play with, but my experience is most projects don't need them. I would rather spend my time implementing functionality than regenerating and recompiling code from proto files. The large projects you mention often have something most projects don't: a well-defined data model and interface that has been heavily iterated and evolved over time. At that point things like backwards compatibility,…

Seconding this. Our team actually decided to implement gRPC + gRPC-gateway in a green-field project at work, as part of an initiative to "PoC" that it could be done and bring us lots of benefits. We got auto-gen'd docs, clients, all the works that gRPC + protobuf in Golang promises.

Between a few breaking upstream changes in grpc-gateway (iirc), learning a new framework vs the stdlib + chi router setup that had already been proven in the greater team, feature shipment dropped through the floor. Couple that with the fact that, as stated, these benefits come with well-iterated data models; conversely, our _not_ well-established data models proved to be a constant PITA when having to regenerate this file and that file. No one used the auto-gen'd docs because they were constantly changing anyway. And integrating with our CI/CD pipeline was a nightmare.

Between providing value with a team's previously proven tools and spending innovation tokens, I'd definitely suggest weighing carefully if you need this to be one of them (bi-directional streaming and a well-established project seem like good candidates though).

Re: REST Servers in Go: Part 1 – standard library

#142

Post by AWebOfBrown »

morelisp wrote: post #128
AWebOfBrown wrote: post #45

Earlier quoted context omitted.

I don't think it's poor to test http handling either, as a coarse grained integration test. The problem I've seen is over-dependence on writing unit tests with mocks instead of biting the bullet and properly testing all the boundaries. I have seen folk end up with 1000+ tests, of which most are useless because the mocks make far too many assumptions, but are necessary because of the layer coupling. This was mostly in…

> I don't think it's poor to test http handling either, as a coarse grained integration test. Sorry to spring a mostly-unrelated question on you about this, but why do you call this an integration test? I recently interviewed three candidates in a row that described their tests in this way, and I thought it was odd, and now I see many people in this thread doing it also. I would call this a functional or behavioral t…

The reality is that I've heard unit, integration and e2e almost entirely used interchangeably, maybe except the former and latter. I don't think trying to nail down the terms to something concrete is necessarily a useful exercise. Attempts to do so, imo, make subjective sense in the terms of the individual's stack/deployment scenario.

To me, it's a contextual term much like 'single responsibility'. In this case, the two "sides" of an integration test are present. A consumer issues a request and a provider responds accordingly. The tests would ascertain that with variations to the client request, the provider behaves in the expected manner.

At which point you might point out that this sounds like an e2e test, but actually using the client web app, for example, might involve far more than a simple http client/library - in no small part because the provider can easily run a simple consumer in memory and avoid the network entirely. E2e tests tend to be far more fragile, so from the perspective of achieving practical continuous deployment, it's a useful distinction.

integration tests in this instance: varying HTTP requests (infrastructure layer) provoke correct behaviour in application layer.

e2e: intended client issues http requests under the correct conditions, which provokes certain provider behaviour, which client then actually utilises correctly.

This, to me, is why the most important part of testing is understanding the boundaries of the tests. Not worrying about their names.

Re: REST Servers in Go: Part 1 – standard library

#143

Post by permille42 »

throwaway894345 wrote: post #120
permille42 wrote: post #105

Earlier quoted context omitted.

There is no "either-or" data type in Golang. That's why. It can only be accomplished by inefficient functional hackery. In C, you just make a struct, have a type present in the struct, and then cast the struct pointer to extended object types to gain additional functionality. In this way you can easily accomplish all sorts of fun things like inheritance. Message passing type designs can easily be accomplished also in…

I think you’re probably trying to make a substantial point but you seem to be mistaken about several things with respect to Go and polymorphism and interfaces such that I can’t figure out what your actual, substantial point is. > Go is a bad language for metaprogramming You’re absolutely right here. > There is no "either-or" data type in Golang. That's why. Correct here too, Go doesn’t have sum types. If you want sum…

Whatever. You obviously don't follow what I am saying at all so why bother.

Re: REST Servers in Go: Part 1 – standard library

#144

Post by permille42 »

morelisp wrote: post #130
throwaway894345 wrote: post #121

Earlier quoted context omitted.

I agree. I’ve seen more prominent figures argue against global state often. Using global state is certainly not idiomatic Go.

> Using global state is certainly not idiomatic Go. Unfortunately it's not this simple (and probably multi-idiomatic). Go definitely adopts more global state than other languages; I don't know any other language that offers a default-global HTTP client and server . Now, part of that is because Go's stdlib goes out of its way to make these appear stateless even though they are not - and this is good, even if you (ofte…

In recent history I have worked a lot with Golang and also encountered the tendency towards globals being the recommended path.

Specifically, I encountered it with logging, frameworks( Gin ), and DBs. It is interesting to me that you called out these three things specifically having encountered them all myself.

I agree also that it is a result of having poor DI support from the language.

I also like your point about PHP users using Golang. The approach Golang uses towards serving web content is very similar to me to how initial PHP frameworks did it.

Go tends to make it easy to setup concurrent things occurring, and so I've found myself spending a fair amount of time speculating on what is thread safe and what isn't.

No matter how much I see it I still do a double take when I see globals in Go modules. Seems like a bad practice to me.

Re: REST Servers in Go: Part 1 – standard library

#145

Post by 0xbkt »

bincyber wrote: post #138
0xbkt wrote: post #96

I would probably go with gRPC + grpc-gateway[1] instead. Declaring your services and models in proto files, annotating your services with google.api.http to help grpc-gateway scaffold your HTTP base. Then just implement your services from the interface generated by grpc-go. You can even register your gRPC services to grpc-gateway without actually bringing up a gRPC server. You finally end up having your exact data mo…

Can you expand on how you can register gRPC services to grpc-gateway without needing to run a gRPC server?

`protoc-gen-grpc-gateway` generates a few public functions which you can use to bind your service to grpc-gateway. You can, - pass a gRPC client instance, - pass a gRPC connection, - pass a gRPC endpoint in string format, - pass a gRPC service instance implementing the service interface generated by `protoc-gen-go-grpc`.

These are meant to fit grpc-gateway to scenarios where gRPC server might be implemented in another programming language, or accepting clients at a TCP port or another kind of a socket, possibly under a different network realm than grpc-gateway. Pretty much like a reverse proxy, but can also be integrated with services in the same code level, escaping the overhead of a transport layer.

Re: REST Servers in Go: Part 1 – standard library

#146

Post by morelisp »

ankurpatel wrote: post #23

Looking at such articles makes me feel we are going back in time rather than improving efficiencies for developers to build RESTful server. If you look at Ruby on Rails you can build the server shown here in one min that is scalable and backed with database. I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic.

> I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic.

It's not about whether you expect Google-like traffic, but about whether you expect a Google-like ratio of traffic/compute to capacity. My day job doesn't deal with "Google-like" traffic (maybe one Google product's worth of traffic), but we also only have about 10 servers to ingest what we do get. My current side project I expect to deal with even fewer orders of magnitude but I also want to be able to run it on a single server to keep costs and operational overhead low.

Re: REST Servers in Go: Part 1 – standard library

#147

Post by omginternets »

permille42 wrote: post #69
omginternets wrote: post #21

Earlier quoted context omitted.

What makes something a "real" DI framework? Wouldn't go.uber.org/fx qualify?

I use "real" in quotes to indicate I am meaning something other than the general meaning of the word "real". What I mean by this is a specific type of DI that I view as effective and a type of meta-programming. Such a DI system does more than just auto-instantiate objects. It also provides a DSL that allows for configuration and ordering of the object instantiation during different phases of system execution. fx does…

I see. Echoing the sibling comment: can you provide a bit more context for the kind of programming that you do? When does such a 'scriptable' DI framework become necessary/preferable? My initial reaction is one of aversion, which makes me think I haven't encountered the kind of problem where scriptable DI solves more problems than it creates.

The closest thing that comes to mind is writing software with some sort of global configuration. i.e.: the user provides some options (CLI flags, environment variables, configuration files, etc.) to change the runtime behavior of the software (e.g. use filesystem storage vs S3 storage, etc.).

In Go, I've always solved this pretty straightforwardly by combining Fx and functional options [1].

[1] https://dave.cheney.net/2014/10/17/functional-options-for-fr...

Re: REST Servers in Go: Part 1 – standard library

#148

Post by sagichmal »

permille42 wrote: post #144
morelisp wrote: post #130

Earlier quoted context omitted.

> Using global state is certainly not idiomatic Go. Unfortunately it's not this simple (and probably multi-idiomatic). Go definitely adopts more global state than other languages; I don't know any other language that offers a default-global HTTP client and server . Now, part of that is because Go's stdlib goes out of its way to make these appear stateless even though they are not - and this is good, even if you (ofte…

In recent history I have worked a lot with Golang and also encountered the tendency towards globals being the recommended path. Specifically, I encountered it with logging, frameworks( Gin ), and DBs. It is interesting to me that you called out these three things specifically having encountered them all myself. I agree also that it is a result of having poor DI support from the language. I also like your point about…

> globals in Go modules. Seems like a bad practice to me.

It is, and good packages don't have them.

Re: REST Servers in Go: Part 1 – standard library

#149

Post by jolux »

sagichmal wrote: post #43
jolux wrote: post #24

Earlier quoted context omitted.

DI is a decoupling technique. You might only have two interfaces to begin with, but the rough idea is that writing to interfaces and using IoC allows you to make many changes by adding code without having to change old code.

Sure, but you don't need a framework or DSL for this in Go.

You don't need it, strictly speaking, in any language. But as applications scale, often an IoC container makes managing lots of dependencies easier.
Post reply on HN