Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't Use Protobuf for Telemetry

richardstartin.github.io

Post reply on HN
Print view View original

161–170 of 195 posts

Re: Don't Use Protobuf for Telemetry

#161

Post by dguaraglia »

rkagerer wrote: post #99
saberience wrote: post #97

Earlier quoted context omitted.

You don't work for Blizzard do you? When I worked there engineers (not my team thankfully) designed their own alternative to protobuf to try and save more bytes over the wire, which in my opinion was a really poor decision. Rather than get on with actually adding value, they ended up pushing back multiple other teams deadlines while adding almost no value. It was a classic "not built here" mentality and doing enginee…

If protobuf works for Google then it essentially works for 99.999% of every other company on the globe. I can't agree with this mindset. Another commenter here pointed out the Google implementation is 20x slower than others, and 1.6MB for this kind of task feels bloaty. Just because it meets Google's needs doesn't mean it's universally adequate.

I think the general point is 'if it works for Google, you should feel relatively safe adopting it for your own use'. It doesn't mean it'll fit every single usecase.

If what you need is something that is super efficient over the wire - or any other such requirement not filled by Protobuf - then maybe look for a different protocol altogether. Or design your own!

But for the rest of us - the '99.999%' - the trade-offs are well understood and we'd rather go for the tool we know than reinvent the wheel or use a less maintained tool.

Re: Don't Use Protobuf for Telemetry

#162

Post by Yeroc »

CamouflagedKiwi wrote: post #15

Is this a thing people care about? From what I've seen of server-side Java applications (we have several at work) 1.6MB and 700 classes are lost in the noise of the endless list of Maven dependencies. I'm sure you can do it a lot more efficiently without using someone else's thing, but what exactly is being optimised for here?

I thought the author addressed this pretty clearly in the article. For his specific use-case he was delivering a JVM agent.

Re: Don't Use Protobuf for Telemetry

#163

Post by rkagerer »

ma2rten wrote: post #118
rkagerer wrote: post #99

Earlier quoted context omitted.

If protobuf works for Google then it essentially works for 99.999% of every other company on the globe. I can't agree with this mindset. Another commenter here pointed out the Google implementation is 20x slower than others, and 1.6MB for this kind of task feels bloaty. Just because it meets Google's needs doesn't mean it's universally adequate.

I did a search on this page and I didn't find the comment you are referencing. It seems very unlikely to me that google's implementation is 20x slower.

See tijsvd's comment about 1 microsecond vs. 50 nanoseconds. (Perhaps I should have caveated that with "in some scenarios")

Re: Don't Use Protobuf for Telemetry

#164

Post by cbsmith »

tijsvd wrote: post #31

I have written an in-house implementation of protobuf for C++ (sorry can't share) and studied the wire format extensively. Google's implementations, at least C++ and Java, are a bunch of bloated crap (or maybe they're very good, but for a use case that I haven't yet encountered). Don't shoot down the format because of a specific implementation, find or write a better one and enjoy the fact that every language has at…

Yeah, the author seems to both understand the nesting issue, and yet not understand how it's not an issue.

As per the format guidelines, you shouldn't have deeply nested structures... or at least if you do it should be in cases where only one or two layers of nesting are decoded at any given stage (I've done this with "envelope" messages).

Of course, there are ways to write serialized lengths of nested elements recursively, you just have to do it yourself. It's just not what you normally want to do, so the standard library and compiler don't emit that logic. If you are going down that road, you gotta ask yourself why.

Re: Don't Use Protobuf for Telemetry

#165

Post by cbsmith »

haberman wrote: post #61
tijsvd wrote: post #54

Earlier quoted context omitted.

Things may have improved since, but the implementations are somehow very large and slow. Things may have changed since, but AFAIK the C++ implementation would always allocate on the heap for nested messages, and perhaps even for optional scalars. This may be optimal for larger documents, but not for smallish messages (my use case was market data and trading instructions). I measured certain small messages, where an e…

> Things may have changed since, but AFAIK the C++ implementation would always allocate on the heap for nested messages This is no longer the case if you use arenas: https://developers.google.com/protocol-buffers/docs/referenc... > and perhaps even for optional scalars This has never been the case, except for string fields where std::string forces us to allocate. Ideally we will eventually use std::string_view for st…

It was never really the case. It's just before arenas it depended on your underlying heap allocator to do all the hard work.

Arenas have been around for a long while now though...

Re: Don't Use Protobuf for Telemetry

#166

Post by cbsmith »

jasonzemos wrote: post #66
tijsvd wrote: post #54

Earlier quoted context omitted.

Things may have improved since, but the implementations are somehow very large and slow. Things may have changed since, but AFAIK the C++ implementation would always allocate on the heap for nested messages, and perhaps even for optional scalars. This may be optimal for larger documents, but not for smallish messages (my use case was market data and trading instructions). I measured certain small messages, where an e…

Since all other comments appear to contradict you and make apologies from authority (after all, Google can't do anything wrong, right?) I'd like to just reassure you with my 20 years of experience developing C (and the last 10 with C++) in network and system software: that any library -- any library -- that forces internal dynamic memory upon its user smells bad. It's not a universal condemnation, but it begs the que…

"that any library -- any library -- that forces internal dynamic memory upon its user smells bad"

Cough.

While with embedded systems there definitely is a big thing about dynamic memory allocation, much as I don't like it, it's not like very popular and successful libraries don't do this. It's a pretty common and accepted practice, and there are standard idioms for how it is done.

Re: Don't Use Protobuf for Telemetry

#167

Post by cbsmith »

vvanders wrote: post #145
kentonv wrote: post #116

Earlier quoted context omitted.

> Constrained embedded systems cover a broad range of things up to and including your phone. No, modern phones are certainly not constrained in the way I meant, and I don't think you could call them "embedded" either. The common programming languages used on phones are very memory-allocation-friendly. > There's few things I hate to see more than a flat profile from memory allocation or cache misses. I think you may b…

> The common programming languages used on phones are very memory-allocation-friendly. I would hardly classify Dalvik or ART as "allocation friendly", they don't perform escape analysis and if you do it constantly you'll be in a world of constant hard GC pauses. Multiple times over the years I've had to build free-lists in Java to avoid this specific problem. Same for C++ if you use one of the built-in generic memory…

> I would hardly classify Dalvik or ART as "allocation friendly", they don't perform escape analysis and if you do it constantly you'll be in a world of constant hard GC pauses. Multiple times over the years I've had to build free-lists in Java to avoid this specific problem.

I don't know what you're doing, but I've generally found free-lists to be a net performance negative in Java libraries. Time and again, I've been called in to "optimize" Java code that uses them, and usually by simply removing them I can get rid of the performance problems entirely.

Re: Don't Use Protobuf for Telemetry

#168

Post by musicale »

vvanders wrote: post #145
kentonv wrote: post #116

Earlier quoted context omitted.

> Constrained embedded systems cover a broad range of things up to and including your phone. No, modern phones are certainly not constrained in the way I meant, and I don't think you could call them "embedded" either. The common programming languages used on phones are very memory-allocation-friendly. > There's few things I hate to see more than a flat profile from memory allocation or cache misses. I think you may b…

> The common programming languages used on phones are very memory-allocation-friendly. I would hardly classify Dalvik or ART as "allocation friendly", they don't perform escape analysis and if you do it constantly you'll be in a world of constant hard GC pauses. Multiple times over the years I've had to build free-lists in Java to avoid this specific problem. Same for C++ if you use one of the built-in generic memory…

You may be right - presumably iPhones have fewer GC pauses (though you can still have VM, loading, compression/decompression, network, and other pauses.)

Be that as it may, lots of people still manage to use Android devices.

Re: Don't Use Protobuf for Telemetry

#169

Post by cbsmith »

haberman wrote: post #44
tijsvd wrote: post #31

I have written an in-house implementation of protobuf for C++ (sorry can't share) and studied the wire format extensively. Google's implementations, at least C++ and Java, are a bunch of bloated crap (or maybe they're very good, but for a use case that I haven't yet encountered). Don't shoot down the format because of a specific implementation, find or write a better one and enjoy the fact that every language has at…

The author of this piece is wanting to stream possibly multi-MB messages with sub-linear memory use. See more in our Twitter conversation here: https://twitter.com/richardstartin/status/134406813297822105...

So, essentially, they're shoving a square peg into a round hole, and are surprised there's a bit of an air gap.

Re: Don't Use Protobuf for Telemetry

#170

Post by cbsmith »

kentonv wrote: post #89
haberman wrote: post #83

Earlier quoted context omitted.

Hey that reminds me, I've always wondered: how did it come to be that delimited encoding "won" for sub-messages and group encoding was deprecated? A lot of these problems would go away if things had gone the other way. Groups can be encoded in one pass, they are efficient to decode (unless you were trying to skip the sub-message, a la LazyField), and they don't have the string/message ambiguity in UnknownFieldSet tha…

That decision predates me. I think it was basically because early versions of protobuf didn't actually support using a message type as field type, so instead people would declare "string" fields and then manually encode/decode another protobuf type into that field. When the ability to explicitly use message types as field types was added to the language, they wanted to use it in those existing protocols without break…

Your recollection sounds about right to me.
Post reply on HN