Live data from Hacker News

Go in Production – Lessons Learned

tdom.dev

91–100 of 109 posts

Re: Go in Production – Lessons Learned

#91
post #72

Earlier quoted context omitted.

It's not really a debate. It divides fairly neatly along the lines of "has been writing Go for a couple of years" vs "has come to Go fairly recently". There's a well-trodden path of developers who were trained in PHP/Rails/JS/Django/etc starting on Go. Their first question is always "what framework should I use?" (you can see this being asked at least once a week on r/golang). They then go down that path, and find (w…

I'm not a backend developer and I can't comment on the actual framework vs no framework in Go discussion. What I want to say it looks a bit arrogant to portray yourself as this enlightenened person who has seen what would take other people years to understand, and to tell them that you won't bother to convince them but they'll understand one day. There has to be a better way to communicate your point.

It took me years too, as I said.

I wrote a whole logging library, and a "better" database access library. I even put them up on Github and asked for feedback in the Go google group. Mostly I got told "you don't need either of these", which I ignored. Until I finally came to understand.

It's a well-trodden path. I'm not the first to walk it, and I see others starting on it now.

Re: Go in Production – Lessons Learned

#92

Earlier quoted context omitted.

I don't feel like this is added by GraphQL because you'll have these questions regardless of using GraphQL. how to cache data (POST is not cacheable) -> You can use GET requests and GET requests are cacheable. how to auth data (anyone has access to everything) -> Authentication or authorization? What do you mean with anyone has access to everything? how to... -> yes?

> You can use GET requests and GET requests are cacheable. GET requests are a crutch added to GraphQL precisely because of limitation of POST requests. And the backend still has to normalise the GET request, and possibly peek inside it to make sure that it is the same as some previous request. > how to auth data (anyone has access to everything) -> Authentication or authorization? What do you mean with anyone has acc…

> GET requests are a crutch added to GraphQL precisely because of limitation of POST requests.

How are GET requests a crutch? If anything GraphQL is completely agnostic to which HTTP method you use to access it. You don't even have to run GraphQL over HTTP, it can work over MQTT, NATS, telnet...

> And the backend still has to normalise the GET request, and possibly peek inside it to make sure that it is the same as some previous request.

Which is what any caching proxy must do anyway?

> Your schema is a single endpoint with all the fields you need exposed. Oh, but a person X with access Y might not have access to fields A, B, C, and D.

In your GraphQL implementation you can just deny fulfilling requests that contain fields person X doesn't have access to. This problem is not limited to GraphQL, it's a generic authorization problem.

> A GraphQL query is ad-hoc. It can have unbounded complexity and unbounded recursion. Ooops, now you have to build complexity analysers and things to figure out recursion levels.

You don't have to build a complexity analyzer or figure out recursion levels, there are already tools that do that for you. But you can go another way and just create a list of approved queries.

> A GraphQL service usually collects data from several external services and/or a database (or even several databases)

Usually? That's just speculation. And that's entirely on the implementation of that service, it has nothing to do with GraphQL spec/technology itself.

Re: Go in Production – Lessons Learned

#93

Earlier quoted context omitted.

> You can use GET requests and GET requests are cacheable. GET requests are a crutch added to GraphQL precisely because of limitation of POST requests. And the backend still has to normalise the GET request, and possibly peek inside it to make sure that it is the same as some previous request. > how to auth data (anyone has access to everything) -> Authentication or authorization? What do you mean with anyone has acc…

> GET requests are a crutch added to GraphQL precisely because of limitation of POST requests. How are GET requests a crutch? If anything GraphQL is completely agnostic to which HTTP method you use to access it. You don't even have to run GraphQL over HTTP, it can work over MQTT, NATS, telnet... > And the backend still has to normalise the GET request, and possibly peek inside it to make sure that it is the same as s…

> How are GET requests a crutch?

They were not in the original spec IIRC. URL's are limited in legth (it's not in the spec, but most clients have a limit) etc.

> Which is what any caching proxy must do anyway?

Nope. A caching proxy can benefit from HTTP Cache Headers [1]. But cache headers don't work well with GraphQL's GET requests, and don't work at all with the default, which is POST.

> This problem is not limited to GraphQL, it's a generic authorization problem.

GraphQL makes it significantly more complex though. Because your requests are ad-hoc.

> You don't have to build a complexity analyzer or figure out recursion levels, there are already tools that do that for you.

Indeed. By adding more and more complexity. And no, tools only solve a part of the problem. Simply a dataloader on a server doesn't entirely solve the N+1 problem.

> But you can go another way and just create a list of approved queries.

Turning it into REST with none of the benefits of REST.

> Usually? That's just speculation. And that's entirely on the implementation of that service

It's not speculation. That's the main use case for GraphQL. But even if you just slap it on top of a single database, you still have the problem of ad-hoc queries hammering your database.

[1] https://www.keycdn.com/blog/http-cache-headers

Re: Go in Production – Lessons Learned

#94
post #4

The more you get to know Go's stdlib the less and less youll think a web framework is necessary. Youll find that utilizing http Round Tripper along with the Handler interface in the http library will make middleware easy. Youll eventually dig into filepath and path methods for extracting path parameters from url paths. And logging and recovery will be a concept youll need to extend outside of just http and into the r…

I see this sentiment a lot in the Go community. I think it is reasonable in some cases, but there are many use cases (vanilla CRUD web apps) where a web framework is really helpful. The standard library is very low level. Want sessions? DIY. Want user auth? DIY. Want CSRF protection? DIY. The list goes on. It feels like a waste of time implementing these "solved problems" from scratch, but the biggest problem is how…

I agree with this. I do believe, that if youre writing a prototypical, client facing, transactional application, a web framework is very useful.

I also advice any individuals coming from Django or friends to use a web framework.

But after some time spent with the stdlib, and understanding how some of those implementations work, it gets to the point where Id rather not read another set of documentation, learn a new mental model, and deal with bugs. This all comes with a framework.

After awhile you realize that the stdlib provides most of what you need, and that writing more vanilla Go can be simpler then learning a full framework.

I will admit, most my work in Go revolves around internal services, and dont deal with web technologies such as CSRF and CORS. So I do acknowledge my opinion here is leaned toward those use cases.

Re: Go in Production – Lessons Learned

#95
post #75
post #59

To those who say frameworks are needed because the stdlib is not enough: No, the stdlib defines very clear interfaces that libraries can implement. The result is that you can drag and drop middlewares from multiple packages into your codebase, because they all conform to the universal middleware signature `func(http.Handler) http.Handler`. Want CSRF? Want Sessions? Want Auth? They all exist as separate packages (chec…

A much needed interface in stdlib is logging.

Yeah I made a library for that. I am not sure if people are actually using it but it got over 70k goddamn hits in my access.log!

Re: Go in Production – Lessons Learned

#96

Earlier quoted context omitted.

> GET requests are a crutch added to GraphQL precisely because of limitation of POST requests. How are GET requests a crutch? If anything GraphQL is completely agnostic to which HTTP method you use to access it. You don't even have to run GraphQL over HTTP, it can work over MQTT, NATS, telnet... > And the backend still has to normalise the GET request, and possibly peek inside it to make sure that it is the same as s…

> How are GET requests a crutch? They were not in the original spec IIRC. URL's are limited in legth (it's not in the spec, but most clients have a limit) etc. > Which is what any caching proxy must do anyway? Nope. A caching proxy can benefit from HTTP Cache Headers [1]. But cache headers don't work well with GraphQL's GET requests, and don't work at all with the default, which is POST. > This problem is not limited…

> They were not in the original spec IIRC. URL's are limited in legth (it's not in the spec, but most clients have a limit) etc.

They were not in spec because the spec doesn't say anything over which medium it should be transported. In fact the spec [1] only mentions the word HTTP 5 times: 4 times in example data and one time discussing implementation details when sending data over HTTP. GraphQL can't be faulted for the limits of the transport over which it is used.

> Nope. A caching proxy can benefit from HTTP Cache Headers [1]. But cache headers don't work well with GraphQL's GET requests, and don't work at all with the default, which is POST.

How do cache headers not work well with GraphQL GET requests? That is entirely up to the server that implements the API. If that server doesn't implement caching well, that's not GraphQL's fault.

> It's not speculation. That's the main use case for GraphQL. But even if you just slap it on top of a single database, you still have the problem of ad-hoc queries hammering your database.

The main use case of GraphQL is any two things that want to exchange data with each other. Merging data from multiple data sources as its main use case is simply not true. The ability of GraphQL to merge different data sources is one of its abilities but it's not intrinsic to GraphQL.

> Turning it into REST with none of the benefits of REST.

And what exactly are those benefits? I'm here defending GraphQL yet none of the downsides of REST are being taken into account. GraphQL brings structure where there was none, that alone is a significant reason to choose GraphQL to structure your API.

> N+1 problem

There are tools like Postgraphile that solve this. It converts your GraphQL query into one efficient database query.

> ad-hoc queries hammering your database

And what prevents anyone from hammering a REST API? GraphQL doesn't release the developer from implementing sane constraints - something that has to happen with any API implementation and not specific to GraphQL.

[1] http://spec.graphql.org/June2018/

Re: Go in Production – Lessons Learned

#97

Earlier quoted context omitted.

It's not really a debate. It divides fairly neatly along the lines of "has been writing Go for a couple of years" vs "has come to Go fairly recently". There's a well-trodden path of developers who were trained in PHP/Rails/JS/Django/etc starting on Go. Their first question is always "what framework should I use?" (you can see this being asked at least once a week on r/golang). They then go down that path, and find (w…

We use standard library in production. Never had a problem with it. It's performance is great, the http.Handler interface is a great pattern too. I rarely find myself straying far from the standard library. If there's something that takes a bit more code but doesn't bring a dependency then that's what I'll do. I definitely agree that there comes a day where you just see the light and realise that simple is good, the…

agree 100%. This is exactly how I feel about it.

Re: Go in Production – Lessons Learned

#99
post #4

The more you get to know Go's stdlib the less and less youll think a web framework is necessary. Youll find that utilizing http Round Tripper along with the Handler interface in the http library will make middleware easy. Youll eventually dig into filepath and path methods for extracting path parameters from url paths. And logging and recovery will be a concept youll need to extend outside of just http and into the r…

> Youll find that utilizing http Round Tripper along with the Handler interface in the http library will make middleware easy.

I thought RoundTripper was a purely client interface - am I missing something? Servers instead have the base Handler, net.Listeners, ConnState, BaseContext, ConnContext etc.

Re: Go in Production – Lessons Learned

#100
Faced similar issues in both of the Go projects I've worked on, particularly the one that used Mongo as the database.

My conclusion from both experiences is that Go is a tool like any other, with its strengths and weaknesses, and with particular idiosyncrasies that make it particularly important to properly architect the application. It can punish you pretty hard[1] if you don't follow proper patterns, or the architecture you decided.

For instance, I think the issues were more pronounced with the project using Mongo precisely because of the flexibility Mongo offers, which allows to delay deciding on schemas, and thus means that the architecture itself can change constantly.

[1]: Such as making coupling and code repetition to balloon pretty fast.

Post reply on HN