Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does gRPC insist on trailers?

carlmastrangelo.com

Post reply on HN
Print view View original

51–60 of 136 posts

Re: Why does gRPC insist on trailers?

#51

Post by zimpenfish »

Matthias247 wrote: post #41

> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure. I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trai…

> It is obvious for the peer if a complete message (inside a stream or single response) is received

If I'm reading [1] correctly, you can't distinguish between [repeated element X is empty] and [message truncated before repeated element X was received] because "A packed repeated field containing zero elements does not appear in the encoded message." You'd need X to be the last part of the message but that's not a problem because "When a message is serialized, there is no guaranteed order [...] parsers must be able to parse fields in any order".

[1] https://developers.google.com/protocol-buffers/docs/encoding...

Re: Why does gRPC insist on trailers?

#52

Post by thayne »

chucky_z wrote: post #30

From my perspective, I think the biggest issue with gRPC is it using HTTP/2. I understand that there’s a lot of reasons to say “No, HTTP/2 is far superior to HTTP/1.1.” However, in terms of proxying _outside Google_ HTTP/2 has lagged, and continues to lags at the L7 proxy layer. I recently performed a lot of high-throughput proxying comparing HAProxy, Traefik, and Envoy. HTTP/1.1 outperformed HTTP/2 (even H2C) by a p…

Also, http/2 over cleartext is not very well supported by a lot of things. Which is probably a good thing when going over the open internet. But it means you have to deal with setting up certificates even if just developing locally, and makes it more difficult to use for IPC on a single host.

Re: Why does gRPC insist on trailers?

#53

Post by alexcpn »

I use GRPC between micro services instead of REST and for that it is really great; All the deficiencies of REST - Non versioned, no typed goes away with GRPC and the protobuf is the official interface for all micro-services. No problems with this approach for over two years now; and also multi language support - We have Go and Java and Python and TypeScript micro-services now happily talking and getting new features and new interface methods updated. Maybe it was demise in the web space; but a jewel in micro-service space

Re: Why does gRPC insist on trailers?

#55

Post by Tyr42 »

teaearlgraycold wrote: post #50
AceJohnny2 wrote: post #32

Offtopic, but: > However, Google is not one single company, but a collection of independent and distrusting companies. This is an important thing to keep in mind when considering the behavior of any large company.

As a Googler, it's worse. Even within one of the "companies" there is distrust in the chain of command.

Googler here. Yeah, it goes my boss, another guy I'm somewhat familiar with, then some cloud of VPs or something and then Sundar.

I have no idea what those vp's are up to and not much faith in their decisions.

Re: Why does gRPC insist on trailers?

#56

Post by arriu »

chucky_z wrote: post #30

From my perspective, I think the biggest issue with gRPC is it using HTTP/2. I understand that there’s a lot of reasons to say “No, HTTP/2 is far superior to HTTP/1.1.” However, in terms of proxying _outside Google_ HTTP/2 has lagged, and continues to lags at the L7 proxy layer. I recently performed a lot of high-throughput proxying comparing HAProxy, Traefik, and Envoy. HTTP/1.1 outperformed HTTP/2 (even H2C) by a p…

Yup, and this is also why many end up proxy gRPC over HTTP/1.1 after giving up on making HTTP/2 work with systems that don’t support it…

Re: Why does gRPC insist on trailers?

#57

Post by e____g »

phendrenad2 wrote: post #42

Quoted post unavailable.

This is very much not it. The Stubby team decided early on to use trailers and was combative and obstinate when the Chrome and GFE (reverse proxy) teams tried to explain why doing so would be a bad idea. The use of trailers in gRPC originates in the hubris of one team, not a conspiracy by Google.

Re: Why does gRPC insist on trailers?

#58

Post by raggi »

twiss wrote: post #48
modeless wrote: post #43

Earlier quoted context omitted.

It's clear that the "single engineer" thing is a lie. Many engineers commented on the Chrome issue with opposing viewpoints, and even the original post describes it being escalated to tech leads on both sides, getting more people involved. I guarantee if it was only one person standing alone opposed to trailers then they would have been overruled. As you say, it's a good thing that Chrome resists adding the pet featu…

Sure, that's fair enough. But I'm not sure if characterizing this feature as a pet feature of the gRPC team is accurate either - after all, it's simply exposing an HTTP 1.1 & H2 feature, and it was already in the WHATWG fetch spec. There, the security concerns were apparently discussed as well [1], and adding the trailer headers as a separate object was deemed safe. I haven't read the entire discussion and don't have…

This could be solved without trailers by piggybacking on top of server sent events rather than requests with trailers, and terminating a stream with status metadata as a final event.

Re: Why does gRPC insist on trailers?

#59

Post by latch »

stefan_ wrote: post #45
Matthias247 wrote: post #39

Earlier quoted context omitted.

The idea here is that http provides something like request/response semantics, methods, path, status code, etc - which are all also useful for gRPC. Websockets provide none of that - they are just message streams. Websockets over http/2 are a new thing, and haven’t even been available at the time gRPC incepted.

That is their whole point. That is why they exist. Dumb reliable pipe, please keep your silly semantics away.

Hardly dumb. They include a variable length, payload masking, their own layer of fragmentation, different message type, validation (txt vs bin) and an extension framework.

Re: Why does gRPC insist on trailers?

#60

Post by Thorrez »

jayd16 wrote: post #35
remram wrote: post #3

> As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. WTF is this? Those are different layer protocols. WebSocket can run on top of HTTP/2. It's like saying TLS is technically superior to TCP, or IP is superior to copper cables. Reference: https://www.rfc-editor.org/rfc/rfc8441.html

HTTP/2 provides features that websockets don't. Even if you were to use websockets over HTTP/2, you'd lose features like being able to multiplex requests _because_ it's a higher level protocol. Why is it wrong to say its better to use a more feature full and lower level protocol?

They provide different APIs. Websockets provide a bidirectional stream of messages. The messages in a given direction are always delivered in order. If they were to be suddenly reordered that would cause a lot of headaches.

While the individual messages can't be multiplexed, different websocket streams over a single HTTP/2 connection can be multiplexed.

I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript.

Post reply on HN