Live data from Hacker News

Why does gRPC insist on trailers?

carlmastrangelo.com

61–70 of 136 posts

Re: Why does gRPC insist on trailers?

#61
post #19
post #4

I had never heard of HTTP trailers. So FYI > The Trailer response header allows the sender to include additional fields at the end of chunked messages in order to supply metadata that might be dynamically generated while the message body is sent, such as a message integrity check, digital signature, or post-processing status. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Tr...

Haven't headers and trailers been renamed in the recent past?

[deleted]

Re: Why does gRPC insist on trailers?

#62
post #19
post #4

I had never heard of HTTP trailers. So FYI > The Trailer response header allows the sender to include additional fields at the end of chunked messages in order to supply metadata that might be dynamically generated while the message body is sent, such as a message integrity check, digital signature, or post-processing status. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Tr...

Haven't headers and trailers been renamed in the recent past?

Yes, slightly - RFC 9110 ("HTTP Semantics") calls them "header fields" and "trailer fields," and it calls "headers" and "trailers" colloquialisms. In a nod to gRPC-style usage, the section on trailer fields even says, "Trailer fields can be useful for supplying...post-processing status information."

https://www.rfc-editor.org/rfc/rfc9110.html#header.fields

https://www.rfc-editor.org/rfc/rfc9110.html#trailer.fields

Re: Why does gRPC insist on trailers?

#63
post #9

Is it still the case that Google Chrome can't support Google gRPC?

To be fair, it's also true that Firefox doesn't expose trailers to the fetch API. To the Chrome and gRPC team's credit, the public Chrome issue actually contains a somewhat substantive back-and-forth; the corresponding Firefox issue has virtually no discussion.

https://bugzilla.mozilla.org/show_bug.cgi?id=1339096

Re: Why does gRPC insist on trailers?

#64
post #47

> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure. I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trai…

He offers two facts that I think explain this well enough: > Additionally, they chose to keep Protobuf as the default wire format, but allow other encodings too. And: > Since streaming is a primary feature of gRPC, we often will not know the length of the response ahead of time. These make sense; you'd enable servers to start streaming back the responses directly as they were generating them, before the length of the…

This doesn't match what I see in the gRPC spec. It says every message must be length-prefixed.

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2....

Disclaimer: I don't know much about gRPC.

Re: Why does gRPC insist on trailers?

#65

> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure. I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trai…

> It is obvious for the peer if a complete message (inside a stream or single response) is received If I'm reading [1] correctly, you can't distinguish between [repeated element X is empty] and [message truncated before repeated element X was received] because "A packed repeated field containing zero elements does not appear in the encoded message." You'd need X to be the last part of the message but that's not a pro…

Yes, the protobuf format makes the end ambiguous, meaning the end needs to be indicated by the protocol containing the protobuf.

But it looks to me like the gRPC spec says that everything must be prefixed by a length at the gRPC layer. So then it doesn't matter that protobuf doesn't internally indicate the end, since the gRPC transport will indicate the end.

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2....

Disclaimer: I don't know much about gRPC.

Re: Why does gRPC insist on trailers?

#66
post #35
post #3

> As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. WTF is this? Those are different layer protocols. WebSocket can run on top of HTTP/2. It's like saying TLS is technically superior to TCP, or IP is superior to copper cables. Reference: https://www.rfc-editor.org/rfc/rfc8441.html

HTTP/2 provides features that websockets don't. Even if you were to use websockets over HTTP/2, you'd lose features like being able to multiplex requests _because_ it's a higher level protocol. Why is it wrong to say its better to use a more feature full and lower level protocol?

TCP can't multiplex. HTTP/2 runs over TCP and does multiplexing.

WebSocket can't multiplex. Nothing prevents gRPC over WebSocket implement multiplexing itself.

Re: Why does gRPC insist on trailers?

#67
post #29

> Why Do We Need Trailers At All? The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive.

> The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive. It did not to me. I would rephrase the argumentation as: - In…

>Shit we realized later that HTTP/1.1 did not necessitate trailers because they could abort the connection and we can not afford to do that anymore

This is an interesting point, but I don't think it's correct. The HTTP/2 spec allows you to send a RST_STREAM frame to indicate that an individual stream had a problem. To be contrasted with an END_STREAM frame that indicates an individual stream ended successfully.

https://www.rfc-editor.org/rfc/rfc7540.html

Re: Why does gRPC insist on trailers?

#68
post #18

Earlier quoted context omitted.

> WebSocket can run on top of HTTP/2 Isn't "websocket" just a standard tcp socket, whose specification to instantiate it was born in a comparatively ephemeral HTTP (of whatever version) request, and which outlives the request, so isn't on top of anything other than tcp?

No, despite the name WebSockets are not plain TCP.

They're an http protocol for setting one up, you mean?

First sentence at this article:

https://en.wikipedia.org/wiki/WebSocket

Re: Why does gRPC insist on trailers?

#69
post #60
post #35

Earlier quoted context omitted.

HTTP/2 provides features that websockets don't. Even if you were to use websockets over HTTP/2, you'd lose features like being able to multiplex requests _because_ it's a higher level protocol. Why is it wrong to say its better to use a more feature full and lower level protocol?

They provide different APIs. Websockets provide a bidirectional stream of messages. The messages in a given direction are always delivered in order. If they were to be suddenly reordered that would cause a lot of headaches. While the individual messages can't be multiplexed, different websocket streams over a single HTTP/2 connection can be multiplexed. I think websockets also provides a feature that HTTP/2 doesn't:…

> I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript.

I will never stop beating this drum: Server-Sent Events and EventSource! Simple to implement, scale, support.

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource

Re: Why does gRPC insist on trailers?

#70
post #18

Earlier quoted context omitted.

No, despite the name WebSockets are not plain TCP.

They're an http protocol for setting one up, you mean? First sentence at this article: https://en.wikipedia.org/wiki/WebSocket

No, they're more than that. After you upgrade to WebSocket you still have to speak the WebSocket protocol over TCP. It includes message framing — with different defined message types, masking, ping/pong, an extension system (including for example optional per-message compression), ...
Post reply on HN