Pyrobuf: A Cython Alternative to Google's Python Protobuf Library

Post by detaro » Sat, Dec 19, 2015, 2:16 PM UTC

diffraction wrote: post #14

slavik81 wrote: post #13

Earlier quoted context omitted.

Perhaps they should do more marketing, because speaking as a C++ developer: I picked between protobufs, flatbuffers and capt'n proto because they were easy to use, had active communities, and they had websites which explained how and why I should use their compiler/library/protocol. When I search the web for information about asn.1, I find very little that is of practical use. What library should I use? Why should I…

you are dismissing an old and proven technology because you feel it is not marketing to you effectively, which is the backend to the global telecom system (ss7) for the past two decades? why is this a movement in computer science to throw out old things that work? this is confusing to me like nosql. but "good marketing" is something that is called cap'n proto, sounds like a joke? if you are curious here is a good blo…

Post by haberman » Sat, Dec 19, 2015, 6:56 PM UTC

pixelmonkey wrote: post #28

haberman wrote: post #21

Earlier quoted context omitted.

It's really interesting to me that the top two comments on this story both wish Protocol Buffers were different, but seem to be asking for opposite things. Your comment wishes that there was a pure-Python implementation that doesn't do metaprogramming. That would imply doing code generation, but having the generated code be more "concrete", so-to-speak (containing all of the actual implementation). However, a differe…

I think we learned a long time ago that though codegen is one path to speedy code, it is not the only path. The thriftpy module dynamically generates a Python module that is based on parsing the thrift schema. That happens once: at module import time. After that, Python has cached the module in sys.modules and it doesn't need to be evaluated again during the program's runtime. The module contains efficient Python cla…

  syntax = "proto3";

  message Test {
    int32 foo = 1;
    double bar = 2;
    Test test = 3;
  }

  require 'google/protobuf'
  
  Google::Protobuf::DescriptorPool.generated_pool.build do
    add_message "Test" do
      optional :foo, :int32, 1
      optional :bar, :double, 2
      optional :test, :message, 3, "Test"
    end
  end

  Test = Google::Protobuf::DescriptorPool.generated_pool.lookup("Test").msgclass

Post by fijal » Sun, Dec 20, 2015, 2:46 PM UTC

haberman wrote: post #32

JulianWasTaken wrote: post #30

Earlier quoted context omitted.

I've certainly observed the same I think :) and yeah pleasing everyone is impossible -- I think though there's a somewhat coherent way to put at least part of those two things together. Not wanting code generation might come from people who don't like the additional build step and artifact deployment. I don't like that either, but what I mean by "less metaprogramming" is that the code that's generated currently makes…

Ah, that's very interesting -- your desire for more "pure" Python comes from a desire for speed. (Some people who wish the generated code was more concrete want this so the generated code is more readable). I am curious what led you to the conclusion that descriptors and metaclasses were making Python protobuf JIT-unfriendly. I ask because most of the "meta" stuff happens at startup, when you first import the module.…