Live data from Hacker News

Using gRPC for (local) inter-process communication (2021)

mpi-hd.mpg.de

41–50 of 99 posts

Re: Using gRPC for (local) inter-process communication (2021)

#41

What I loved about Fuchsia was its IPC interface, using FIDL which is like a more optimized version of protobufs. https://fuchsia.dev/fuchsia-src/get-started/learn/fidl/fidl

loved - in the past tense?

fuchsia was pretty deeply impacted by google layoffs iirc

Re: Using gRPC for (local) inter-process communication (2021)

#42
post #36
post #32

Earlier quoted context omitted.

> Ok, now it sounds like you're blaming unix sockets for someone's shitty code... No, I am just saying that the unix socket is not Brawndo (or maybe it is?), it does not necessarily have what IPCs crave. Sprinkling it into your architecture may or may not be relevant to the efficiency and performance of the result.

Sorry, what's brawndo? (Searching only gives me movie results?) We started out discussing AF_UNIX vs. AF_INET6. If you can conceptually use something faster than sockets that's great, but if you're down to a socket, unix domain will generally beat inet domain...

it's what plants crave! it's got electrolytes.

Re: Using gRPC for (local) inter-process communication (2021)

#43
post #21
post #12

> Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. That is exactly what COM/WinRT, XPC, Android Binder, D-BUS are. Naturally they have several optimisations for local execution.

Binder seriously underappreciated, IMHO. But I think it makes sense to use gRPC or something like it if there is any possibility that in the future an "IPC" will become an "RPC" to a foreign host. You don't want to be stuck trying to change an IPC into an RPC if it was foreseeable that it would eventually become remote due to scale.

In my mind the abstraction should allow for RPCs and being on the same machine should allow to optimise things a bit, this way you simply build for the general case and lose little to no performance.

Think of the loopback, my programs don't know (or at least shouldn't know) that IPs like 127.0.0.5 are special, but then the kernel knows that messages there are not going to go on any wire and handles that differently.

Re: Using gRPC for (local) inter-process communication (2021)

#44
post #12

> Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. That is exactly what COM/WinRT, XPC, Android Binder, D-BUS are. Naturally they have several optimisations for local execution.

Btw. Modern windows also superports Unix domain sockets, so if you have an app that has another service that will run on the same machine or on a different one it is not so bad to use grpc over uds.

Re: Using gRPC for (local) inter-process communication (2021)

#45
post #44
post #12

> Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. That is exactly what COM/WinRT, XPC, Android Binder, D-BUS are. Naturally they have several optimisations for local execution.

Btw. Modern windows also superports Unix domain sockets, so if you have an app that has another service that will run on the same machine or on a different one it is not so bad to use grpc over uds.

Nice idea, although it is still slower than COM.

COM can run over the network (DCOM), inside the same computer on its own process (out-proc), inside the client (in-proc), designed for in-proc but running as out-proc (COM host).

So for max performance, with the caveat of possibly damaging the host, in-proc will do it, and be faster than any kind of sockets.

Re: Using gRPC for (local) inter-process communication (2021)

#46
post #37

Earlier quoted context omitted.

C++ generated code from protobuf/grpc is pretty awful in my experience.

Do you need to look at that generated code though? I haven't used gRPC yet (some poor historical decisions mean I can't use it in my production code so I'm not in a hurry - architecture is rethinking those decisions in hopes that we can start using it so ask me in 5 years what I think). My experience with other generated code is that it is not readable but you never read it so who cares - instead you just trust the i…

I meant the interfaces are horrible. As you said, as long as it has a good interface and good performance, I wouldn't mind.

For example, here's the official tutorial for using the async callback interfaces in gRPC: https://grpc.io/docs/languages/cpp/callback/

It encourages you to write code with practices that are quite universally considered bad in modern C++ due to a very high chance of introducing memory bugs, such as allocating objects with new and expecting them to clean themselves up via delete this;. Idiomatic modern C++ would be using smart pointers, or go a completely different route with co-routines and no heap-allocated objects.

Re: Using gRPC for (local) inter-process communication (2021)

#47
post #40

Earlier quoted context omitted.

JSON contains a description of the structure of the data that is readable by both machines and humans. JSON can certainly go wrong but it’s much simpler to see when it has because of this. GRPC is usually a binary black box that adds loads of developer time to upskill, debug, figure out error cases and introduces whole new classes of potential bugs. If you are building something that needs binary performance that GRP…

> JSON contains a description of the structure of the data that is readable by both machines and humans. No, it by definition does not, because JSON has no schema. Only your application contains and knows the (expected) structure of the data, but you literally cannot know what structure any random blob of JSON objects will have without a separate schema. When you read a random /docs page telling you "the structure of…

If I gave you a JSON object with name, age, position, gender etc. etc. would you not say it has structure? If I give you a GRPC binary you need the separate schema and tools to be able to comprehend it. That’s all I’m saying is the separation of the schema from some minimal structure makes the debugging of services more difficult. I would also add the GRPC implementation I used in Javascript (long ago) was not actually checking the types of the field in a lot of cases so rather than being a schema that rejects if some field is not a text field it would just return binary junk. JSON Schema or almost anything else will give you a parsing error instead.

Maybe the tools are fantastic not but I still think being able to debug messages without them is an advantage in almost all systems, you probably don’t need the level of performance GRPC provides.

If you’re using JSON Protobufs why would you add this extra complexity - it will mean messaging is just as slow as using JSON. What are the core advantages of GRPC under these conditions?

Re: Using gRPC for (local) inter-process communication (2021)

#48
post #37

Earlier quoted context omitted.

Do you need to look at that generated code though? I haven't used gRPC yet (some poor historical decisions mean I can't use it in my production code so I'm not in a hurry - architecture is rethinking those decisions in hopes that we can start using it so ask me in 5 years what I think). My experience with other generated code is that it is not readable but you never read it so who cares - instead you just trust the i…

I meant the interfaces are horrible. As you said, as long as it has a good interface and good performance, I wouldn't mind. For example, here's the official tutorial for using the async callback interfaces in gRPC: https://grpc.io/docs/languages/cpp/callback/ It encourages you to write code with practices that are quite universally considered bad in modern C++ due to a very high chance of introducing memory bugs, suc…

ouch. I'm temped to look up the protocol and build my own grpc implementation from scratch. Generated code isn't that hard to create, but it is something I'd expect to be able to just use someone else's version instead of writing (and supporting) myself.

Re: Using gRPC for (local) inter-process communication (2021)

#49
post #45
post #44

Earlier quoted context omitted.

Btw. Modern windows also superports Unix domain sockets, so if you have an app that has another service that will run on the same machine or on a different one it is not so bad to use grpc over uds.

Nice idea, although it is still slower than COM. COM can run over the network (DCOM), inside the same computer on its own process (out-proc), inside the client (in-proc), designed for in-proc but running as out-proc (COM host). So for max performance, with the caveat of possibly damaging the host, in-proc will do it, and be faster than any kind of sockets.

> COM can run over the network (DCOM)

Ah the good ol' Blaster worm...

https://en.wikipedia.org/wiki/Blaster_(computer_worm)

Re: Using gRPC for (local) inter-process communication (2021)

#50
> In our scenario of local IPC, some obvious tuning options exist: data is exchanged via a Unix domain socket (unix:// address) instead of a TCP socket

AFAIK at least on linux there is no difference between using a UDS and a tcp socket connected to localhost.

Post reply on HN