Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gRPC: From service definition to wire format

kreya.app

Post reply on HN
Print view View original

21–30 of 33 posts

Re: gRPC: From service definition to wire format

#21

Post by smnscu »

selfmodruntime wrote: post #20
kjuulh wrote: post #3

Earlier quoted context omitted.

I think connect-rpc[0] strikes a good balance between normal http apis, and gRPC. It allows protobuf as json. So you could think of it as an opinionated http api spec. A health check would be just a call to an url /something.v1.MyService/MyMethod -d { "input": "something }. it works really well, and the tooling is pretty good, though it isn't that widely supported yet. Rust for one doesn't have an implementation. But…

+1 for connect-rpc. After go-micro‘s maintainer went off the rails, I ripped it all out and switched to connect.

> After go-micro‘s maintainer went off the rails

What do you mean by this? Genuinely curious, as someone who's followed that project in the past.

Re: gRPC: From service definition to wire format

#22

Post by aanet »

I’m old enough to remember the days of CORBA (Orbix, Iona, BEA, anyone ??) and its IDL, the IDL compiler, stubs and other props and doodads. Ah yes, and the registry as well, where the services registered themselves, and the discovery mechanisms.

gRPC (a very Googly thing) took it all, hook line and sinker and made it URLesque

Can’t recall how the ORB overhead has been resolved in gRPCs

Re: gRPC: From service definition to wire format

#23

Post by speedbird »

aanet wrote: post #22

I’m old enough to remember the days of CORBA (Orbix, Iona, BEA, anyone ??) and its IDL, the IDL compiler, stubs and other props and doodads. Ah yes, and the registry as well, where the services registered themselves, and the discovery mechanisms. gRPC (a very Googly thing) took it all, hook line and sinker and made it URLesque Can’t recall how the ORB overhead has been resolved in gRPCs

And still I found it easier to use than gRPC.

Re: gRPC: From service definition to wire format

#24

Post by asim »

smnscu wrote: post #21
selfmodruntime wrote: post #20

Earlier quoted context omitted.

+1 for connect-rpc. After go-micro‘s maintainer went off the rails, I ripped it all out and switched to connect.

> After go-micro‘s maintainer went off the rails What do you mean by this? Genuinely curious, as someone who's followed that project in the past.

He probably means when I took VC funding in 2019 and started to rip apart the framework to try build a platform and business. The 2-3 years after were very chaotic.

My goal was never to serve the community but instead leverage it to build a business. Ultimately that failed. The truth is it's very difficult to sustain open source. Go-micro was never the end goal. It was always a stepping stone to a platform e.g microservices PaaS. A lot of hard lessons learned along the way.

Now with Copilot and AI I'm able to go back and fix a lot of issues but nothing will fix trust with a community or the passage of time. People move on. It served a certain purpose at certain time.

Note: The company behind connect-rpc raised $100m but for more of a build system around protobuf as opposed to the rpc framework but this was my thinking as well. The ability to raise $10-20m would create the space to build the platform off the back of the success of the framework.

Re: gRPC: From service definition to wire format

#25

Post by PessimalDecimal »

dewey wrote: post #4
tempest_ wrote: post #2

I like the idea of grpc because I wanted the contract but I tried it on a small service and I think I would avoid it in the future. Too many rough edges and features I didnt really need. I was using it in Rust and python mainly (maybe it is better in Go?) but it had a whole bunch of google stuff in there I didnt need. - Configuring the python client with a json string that did not seem to have a documented schema - E…

I've only enjoyed using Protobuf + gRPC after we've started using https://buf.build . Before that it was always a pain with Makefiles building obscure commands, then developers having different versions of the Protobuf compiler installed and all kinds of paper cuts like that. Now it's just "buf generate", every developer has the exact same settings defined in the repo and on the frontend side we are just importing th…

buf.build sounds interesting as a middle ground for using protos without going all-in on the Bazel build ecosystem.

Re: gRPC: From service definition to wire format

#26

Post by PessimalDecimal »

est wrote: post #7

> The contract-first philosophy gRPC/protobuf is largely a Google cult. I've seen too projects with complex business logic simply give up and embed JSON strings inside pb. Like WTF...? Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly. As it occured to me, gRPC could optionally just serve those .pr…

> Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly.

Is this an issue with protobufs per se though? It's a data schema. How are people supposed to develop to a shared schema if a team doesn't - you know - share their schema? That could happen with any other particular choice for how schemas are defined.

Re: gRPC: From service definition to wire format

#27

Post by neRok »

tempest_ wrote: post #2

I like the idea of grpc because I wanted the contract but I tried it on a small service and I think I would avoid it in the future. Too many rough edges and features I didnt really need. I was using it in Rust and python mainly (maybe it is better in Go?) but it had a whole bunch of google stuff in there I didnt need. - Configuring the python client with a json string that did not seem to have a documented schema - E…

> - Configuring the python client with a json string that did not seem to have a documented schema

I'm far from an expert, yet I came to believe that what you've described is basically "code smell". And the smell probably comes from seemingly innocuous things like enum's.

And you wondered if the solution was using Go, but no, it isn't. I was actually Go at the time myself (this was a few years ago, and I used Twirp instead of Protobuf) - but I realised that RDBMS > "Server(Go)" layer had quirks, and then the "Server(Go)" > "API(JS)" had other quirks -- and so I realised that you may as well "splat" out every attribute/relationship. Because ultimately, that's the problem...

Eg: is it a null field, or undefined, or empty, or false, or [], or {}? ...

[] == my valentines day inbox. :P

Re: gRPC: From service definition to wire format

#28

Post by jeffbee »

aanet wrote: post #22

I’m old enough to remember the days of CORBA (Orbix, Iona, BEA, anyone ??) and its IDL, the IDL compiler, stubs and other props and doodads. Ah yes, and the registry as well, where the services registered themselves, and the discovery mechanisms. gRPC (a very Googly thing) took it all, hook line and sinker and made it URLesque Can’t recall how the ORB overhead has been resolved in gRPCs

There is nothing analogous to the ORB in gRPC, so your complaint seems hallucinated. Callers are entirely responsible for figuring out where their services are. gRPC holds no opinions whatsoever about how your servers are named or how their lifecycles are managed. There is not a "bus".

Re: gRPC: From service definition to wire format

#29

Post by jeffbee »

pm90 wrote: post #17

My gripe with grpc is that it doesn’t play super well with kubernetes services… you have to take a little bit of care, you need to understand how k8s services work and you have to understand how load balancing in grpc works. Ideally I would want to use protobuf as an interchange format, and a “dumb” http server that understands that. That being said… once you do configure it properly it can be a powerful tool. The co…

If you just want to send a protobuf to a host:port there's no reason you can't do that with gRPC. Client load balancers are something you can optionally layer on top.

Re: gRPC: From service definition to wire format

#30

Post by ragall »

PessimalDecimal wrote: post #26
est wrote: post #7

> The contract-first philosophy gRPC/protobuf is largely a Google cult. I've seen too projects with complex business logic simply give up and embed JSON strings inside pb. Like WTF...? Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly. As it occured to me, gRPC could optionally just serve those .pr…

> Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly. Is this an issue with protobufs per se though? It's a data schema. How are people supposed to develop to a shared schema if a team doesn't - you know - share their schema? That could happen with any other particular choice for how schemas are def…

It's a problem with PB because it requires everything to be typed (unless you use Any), which requires all middleware to eagerly type check all data passing through. With JSON, validation will be typically done only by the endpoints, which allows for much faster development.

There was a blog a few years ago, where an engineer working on the Google Cloud console was complaining that simply adding a checkbox to one of the pages required modifying ~20 internal protos and 6 months of rollout. That's an obvious downside that I wish I knew how to fix.

Post reply on HN