Live data from Hacker News

Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

cloud.google.com

231–240 of 286 posts

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#231

I've been building API's for a long time, using gRPC, and HTTP/REST (we'll not go into CORBA or DCOM, because I'll cry). To that end, I've open sourced a Go library for generating your clients and servers from OpenAPI specs ( https://github.com/oapi-codegen/oapi-codegen ). I disagree with the way this article breaks down the options. There is no difference between OpenAPI and REST, it's a strange distinction. OpenAPI…

[deleted]

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#232

I've been building API's for a long time, using gRPC, and HTTP/REST (we'll not go into CORBA or DCOM, because I'll cry). To that end, I've open sourced a Go library for generating your clients and servers from OpenAPI specs ( https://github.com/oapi-codegen/oapi-codegen ). I disagree with the way this article breaks down the options. There is no difference between OpenAPI and REST, it's a strange distinction. OpenAPI…

There is a distinction between (proper) REST and what this blog calls "OpenAPI". But the thing is, almost no one builds a true, proper REST API. In practice, everyone uses the OpenAPI approach. The way that REST was defined by Roy Fielding in his 2000 Ph.D dissertation ("Architectural Styles and the Design of Network-based Software Architectures") it was supposed to allow a web-like exploring of all available resourc…

> Pedants (which let's face it, most of us are) will often describe what is done in practice as "RESTful" rather than "REST" just to acknowledge that they are not implementing Fielding's definition of REST.

Yes, exactly. I've never actually worked with any group whom had actually implemented full REST. When working with teams on public interface definitions I've personally tended to use the so-called Richardson's Maturity Model[0] and advocated for what it calls 'Level 2', which is what I think most of us find rather canonical and principal of least surprise regarding a RESTful interface.

[0] - https://en.wikipedia.org/wiki/Richardson_Maturity_Model

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#233

Earlier quoted context omitted.

IMO the problem with gRPC isn't the protocol or the protobufs, but the terrible tooling - at least on the Java end. It generates shit code with awful developer ergonomics. When you run the protobuf builder... * The client stub is a concrete final class. It can't be mocked in tests. * When implementing a server, you have to extend a concrete class (not an interface). * The server method has an async method signature.…

Protobuf is an atrocious protocol. Whatever other problems gRPC has may be worse, but Protobuf doesn't make anything better that's for sure. The reason to use it may be that you are required to by the side you cannot control, or this is the only thing you know. Otherwise it's a disaster. It's really upsetting that a lot of things used in this domain are the first attempt by the author to make something of sorts. So m…

Agree. As an example, this proto generates 584 lines of C++, links to 173k lines of dependencies, and generates a 21Kb object file, even before adding grpc:

syntax = "proto3"; message LonLat { float lon = 1; float lat = 2; }

Looking through the generated headers, they are full of autogenerated slop with loads of dependencies, all to read a struct with 2 primitive fields. For a real monorepo, this adds up quickly.

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#234

Earlier quoted context omitted.

IMO the problem with gRPC isn't the protocol or the protobufs, but the terrible tooling - at least on the Java end. It generates shit code with awful developer ergonomics. When you run the protobuf builder... * The client stub is a concrete final class. It can't be mocked in tests. * When implementing a server, you have to extend a concrete class (not an interface). * The server method has an async method signature.…

Protobuf is an atrocious protocol. Whatever other problems gRPC has may be worse, but Protobuf doesn't make anything better that's for sure. The reason to use it may be that you are required to by the side you cannot control, or this is the only thing you know. Otherwise it's a disaster. It's really upsetting that a lot of things used in this domain are the first attempt by the author to make something of sorts. So m…

[deleted]

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#235
post #145

Earlier quoted context omitted.

For a public API I wouldn’t do this, but for private APIs we just do POST /api/doThingy with a JSON body, easy peasy RPC anyone can participate in with the most basic HTTP client. Works great on every OS and in every browser, no fucking around with “what goes in the URL path” vs “what goes in query params” vs “what goes in the body”. You can even do this with gRPC if you’re using Buf or Connect - one of the server th…

I'd argue just making everything POST is the correct way to do a public Api too. REST tricks you into endpoints no one really wants, or you break it anyway to support functionality needed. SOAP was heavy with it's request/respone, but it was absolutely correct that just sending everything as POST across the wire is easier to work with.

Yeah, I like doing this as well. And all the data goes in the request body. No query parameters.

Especially when the primary intended client is an SPA, where the URL shown is decoupled with the API URL.

Little bit of a memory jolt: I once built a (not for prod) backend in python as follows:

write a list of functions, one for each RPC, in a file `functions.py`

then write this generic function for flask:

  import server.functions as functions

  @server.post("/")
  def api(method: str):
      data: Any = request.json if request.is_json else {}

      fn = lookup(functions, method)
      if fn is None:
          return {"error": "Method not found."}
      return fn(data)

And `lookup()` looks like:

  def lookup(module: ModuleType, method: str):
      md = module.__dict__
      mn = module.__name__
      is_present = method in md
      is_not_imported = md[method].__module__ == mn
      is_a_function = inspect.isfunction(md[method])

      if is_present and is_not_imported and is_a_function:
          return md[method]
      return None
So writing a new RPC is just writing a new function, and it all gets automatically wired up to `/api/function_name`. Quite nice.

The other nice feature there was automatic "docs" generation, from the python docstring of the function. You see, in python you can dynamically read the docstring of an object. So, I wrote this:

  def get_docs(module: ModuleType):
      md = module.__dict__
      mn = module.__name__
      docs = ""

      for name in md:
          if not inspect.isfunction(md[name]) or md[name].__module__ != mn:
              continue
          docs += md[name].__doc__ + "\n
\n" return docs[:-6]
Gives a simple text documentation which I served at an endpoint. Of course you could also write the docstring in openapi yaml format and serve it that way too.

Quite cursed overall, but hey, its python.

One of the worst footguns here is that you could accidentally expose helper functions, so you have to be sure to not write those in the functions file :P

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#236
post #81

Earlier quoted context omitted.

> No it isn't. Evidence: I'm reading this in a web browser. And you might not that this site is _not_ REST-ful. It's certainly HTTP, but not REST. > Bikeshedding the spelling of resource identifiers? Or what "verb" should be used to express specialized domain semantics? Or whether we want to use If-Modified-Since header or explicitly specify the condition in the JSON body. And 6 months later, with some people asking…

> It's certainly HTTP, but not REST. How isn't it RESTful? It's a single entrypoint using content types to tell the client how to interpret it, and with exploratory clues to other content in the website.

The "R" letter means "Representational". It requires a certain style of API. E.g. instead of "/item?id=23984792834" you have "/items/comments/23984792834".

HN doesn't have this.

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#237

Earlier quoted context omitted.

IMO the problem with gRPC isn't the protocol or the protobufs, but the terrible tooling - at least on the Java end. It generates shit code with awful developer ergonomics. When you run the protobuf builder... * The client stub is a concrete final class. It can't be mocked in tests. * When implementing a server, you have to extend a concrete class (not an interface). * The server method has an async method signature.…

Protobuf is an atrocious protocol. Whatever other problems gRPC has may be worse, but Protobuf doesn't make anything better that's for sure. The reason to use it may be that you are required to by the side you cannot control, or this is the only thing you know. Otherwise it's a disaster. It's really upsetting that a lot of things used in this domain are the first attempt by the author to make something of sorts. So m…

[deleted]

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#238
post #48

Earlier quoted context omitted.

> GraphQL is even better just a casual sentence at the end? How about no. It's in the name, a query-oriented API, useless if you don't need flexible queries. Why don't you address the problem they talked about, what is the cli tool I can use to test grpc, what about gui client?

> a query-oriented API, useless if you don't need flexible queries Right but, the typical web service at the typical startup does need flexible queries. I feel people both overestimate its implications and under estimate its value. - Standard "I need everything" in the model call - Simplified "I need two properties call", like id + display name for a dropdown - I need everything + a few related fields, which maybe re…

GraphQL is fine until you have enough data to care about performance, at which point you have to go through and figure out where some insane SQL is coming from, which ultimately is some stitched together hodgepodge of various GraphQL query types, which maybe you can build some special indexes to support or maybe you have to adjust what's being queried. Either way, you patch that hole, and then a month later you have a new page that's failing to load because it's generating a query that is causing your DB CPU to jump to 90%.

I'm convinced at this point that GraphQL only works effectively at a small scale, where inefficient queries aren't disastrously slow/heavy, OR at a large enough scale where you can dedicate at least an entire team of engineers to constantly tackle performance issues, caching, etc.

To me it also makes no sense at startups, which don't generally have such a high wall between frontend and backend engineering. I've seen it used at two startups, and both spent way more time on dealing with GraphQL BS than it would have taken to either ask another team to do query updates or just learn to write SQL. Indeed, at $CURRENT_JOB the engineering team for a product using GraphQL actively pushed for moving away from it and to server-side rendering with Svelte and normal knex-based SQL queries, despite the fact that none of them were backend engineers by trade. The GraphQL was just too difficult to reason about from a performance perspective.

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#239
post #145

Earlier quoted context omitted.

For a public API I wouldn’t do this, but for private APIs we just do POST /api/doThingy with a JSON body, easy peasy RPC anyone can participate in with the most basic HTTP client. Works great on every OS and in every browser, no fucking around with “what goes in the URL path” vs “what goes in query params” vs “what goes in the body”. You can even do this with gRPC if you’re using Buf or Connect - one of the server th…

This. The amount of time lost debating correct rest semantics for a use case is staggering.

Yeah, when it matters in close to 0% of cases. Everyone reads the docs for everything anyways, any shared knowledge granting implicit meaning to things is very close to useless in practice with REST APIs.

Re: Understanding gRPC, OpenAPI and REST and when to use them in API design (2020)

#240

Earlier quoted context omitted.

> According to this, what is GraphQL? GraphQL is akin to gRPC: a non-HTTP protocol tunnelled over HTTP. Unlike gRPC, I’m unconvinced that GraphQL is ever really a great answer. I think what the latter does can be done natively in HTTP.

For all the people singing the praises of how efficient gRPC is, I enjoy countering that the most efficient response is one which doesn't include 99% of data that the client doesn't care about in the slightest GCP (and I believe Azure, too) offer `GET /thing?$fields=alpha,beta.charlie` style field selection but now there's a half-baked DSL in a queryparam and it almost certainly doesn't allow me to actually express w…

Efficient in terms of wire transfer sure, but GraphQL tends to wind up generating queries that are quite difficult to optimize at the DB layer, so you wind up spending way more computer and time than you would otherwise need. If you're in an organization where folks with no database knowledge are writing the GraphQL queries, this winds up being a never-ending game of whack-a-mole. For anything performance sensitive, I'd much rather have a nice, optimized endpoint that returns more data than the client needs rather than have the client be able to issue any query they want.
Post reply on HN