Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Protobuf-py: Protobuf for Python, without compromises

buf.build

Post reply on HN
Print view View original

11–20 of 65 posts

Re: Protobuf-py: Protobuf for Python, without compromises

#11

Post by usrnm »

crabbone wrote: post #8
usrnm wrote: post #4

After gogoproto I'm hesitant to depend on another non-standard implementation, getting off gogo was a pain. This thing may be better than the one from Google (gogo definitely was), but can we be sure that it will still be around in 10 years?

Hey. I wrote another Python implementation of Protobuf. (protopy https://gitlab.com/doodles-archive/protopy it was a while ago and haven't touched it since). I'm not saying it's better than whatever this is or that it's any good, I just post it as a proof of sorts that I'm familiar with the problem. So, without further ado: Protobuf isn't a standard. You can't have a non-standard implementation of something that does…

At least, Google has the resources and the will to support all their implementations in the long run. I don't always agree with what they do there, but at least I can be sure that it will still work ten years from now. At some point it becomes more important than implementation details

Re: Protobuf-py: Protobuf for Python, without compromises

#12

Post by newswangerd »

This is incredible news! I’ve used protobuf in Python, Go, Kotlin and Dart and the Python implementation is totally unusable. I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect. I’ve been waiting for a proper python implementation for years now!

Re: Protobuf-py: Protobuf for Python, without compromises

#13

Post by newswangerd »

usrnm wrote: post #4

After gogoproto I'm hesitant to depend on another non-standard implementation, getting off gogo was a pain. This thing may be better than the one from Google (gogo definitely was), but can we be sure that it will still be around in 10 years?

Buf is well established and maintains a lot of protobuf packages for many languages, including the YAML implementation for go.

Re: Protobuf-py: Protobuf for Python, without compromises

#14

Post by StilesCrisis »

crabbone wrote: post #8
usrnm wrote: post #4

After gogoproto I'm hesitant to depend on another non-standard implementation, getting off gogo was a pain. This thing may be better than the one from Google (gogo definitely was), but can we be sure that it will still be around in 10 years?

Hey. I wrote another Python implementation of Protobuf. (protopy https://gitlab.com/doodles-archive/protopy it was a while ago and haven't touched it since). I'm not saying it's better than whatever this is or that it's any good, I just post it as a proof of sorts that I'm familiar with the problem. So, without further ado: Protobuf isn't a standard. You can't have a non-standard implementation of something that does…

> I just think that Protobuf is not a good format for writing reliable software that aims for decades of usage.

I am not a fan of Protobuf at all, but it's already demonstrated its ability to ship extremely reliable software with multi-decade lifespans. It's one of the few things Google _hasn't_ deprecated, and it's the backbone of the search and ads stack.

Re: Protobuf-py: Protobuf for Python, without compromises

#15

Post by lyu07282 »

newswangerd wrote: post #13
usrnm wrote: post #4

After gogoproto I'm hesitant to depend on another non-standard implementation, getting off gogo was a pain. This thing may be better than the one from Google (gogo definitely was), but can we be sure that it will still be around in 10 years?

Buf is well established and maintains a lot of protobuf packages for many languages, including the YAML implementation for go.

Going strong since 2019! It's a wonderful company entirely dedicated to making google's miserable protobuf "somewhat" useable.

Re: Protobuf-py: Protobuf for Python, without compromises

#16

Post by fmbb »

crabbone wrote: post #8
usrnm wrote: post #4

After gogoproto I'm hesitant to depend on another non-standard implementation, getting off gogo was a pain. This thing may be better than the one from Google (gogo definitely was), but can we be sure that it will still be around in 10 years?

Hey. I wrote another Python implementation of Protobuf. (protopy https://gitlab.com/doodles-archive/protopy it was a while ago and haven't touched it since). I'm not saying it's better than whatever this is or that it's any good, I just post it as a proof of sorts that I'm familiar with the problem. So, without further ado: Protobuf isn't a standard. You can't have a non-standard implementation of something that does…

Why does it matter for some Python implementation if the Google C++ implementation has a lazy or eager parser?

The important part of protobuf is the spec of the wire format. That is what makes the standard an interop format.

Personally I also prefer code generation over dynamic parsers and generators. This is not an idiosyncrasy of C++, it is just the objectively good way to handle IDLs regardless of programming language.

Re: Protobuf-py: Protobuf for Python, without compromises

#17

Post by masklinn »

newswangerd wrote: post #12

This is incredible news! I’ve used protobuf in Python, Go, Kotlin and Dart and the Python implementation is totally unusable. I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect. I’ve been waiting for a proper python implementation for years now!

> I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect.

TFA seems to say that they’re just thin proxies over the underlying C++ APIs, which would more than do it, and does not surprise me (the re2 Python bindings are similar, not as bad since they don’t generate Python code but they’re really c++-y — in Google’s flavour too — and uncomfortable).

Re: Protobuf-py: Protobuf for Python, without compromises

#18

Post by Intralexical »

usrnm wrote: post #7
didip wrote: post #5

Earlier quoted context omitted.

What was the backstory of gogoproto?

The golang implementation of protobuf sucked historically (still does, but is improving) and gogo was an alternative that fixed a lot of problems and was nicer overall. Until its creator burned out and deprecated it. Chasing a constantly moving target that you have no control over is very taxing in the long run.

This is concerning to hear. What do Protobufs accomplish, that requires them to be a constantly moving target?

Re: Protobuf-py: Protobuf for Python, without compromises

#20

Post by arccy »

Intralexical wrote: post #18
usrnm wrote: post #7

Earlier quoted context omitted.

The golang implementation of protobuf sucked historically (still does, but is improving) and gogo was an alternative that fixed a lot of problems and was nicer overall. Until its creator burned out and deprecated it. Chasing a constantly moving target that you have no control over is very taxing in the long run.

This is concerning to hear. What do Protobufs accomplish, that requires them to be a constantly moving target?

https://www.youtube.com/watch?v=HTIltI0NuNg

I think it's less that protobuf is a moving target, and more that gogo tried to add in all the features that google didn't want to maintain, and learned that maintaining a massive feature matrix was impossible.

Post reply on HN