41–50 of 78 posts
I'm not sure that the phrasing in the article is particularly fair: > The maintainers of Gogo, understandably, were not up to the gigantic task. I'm 99% sure they are "up to" (as in "capable of") doing so, they are just not "up for" it (as in, "will not do it").
Earlier quoted context omitted.
> That's what this whole thread is about: you can literally do just that I don't know how you get that from the thread: > Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. > If you are willing to use cgo, google already implemented one for gapid. > there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to…
You are misunderstanding the thread, I just mentioned some of the languages I like (still waiting for Go's generics), and the comment I was replying to made an assert about an implementation that uses cgo. Both of us are dismissing the assertion that "Arenas are, however, unfeasible to implement in Go because it is a garbage collected language." You can do manually memory allocation via a syscall into the host OS, us…
Earlier quoted context omitted.
I can't believe we've managed to have this lengthy of a discussion about GC languages and speed without anyone mentioning rust. Has HN turned a corner?
Rust has an arena allocator too[1], but it is implemented with 165(!!!) usages of unsafe. :) [1] https://github.com/fitzgen/bumpalo
From the same author, a zero-unsafe arena allocator: https://github.com/fitzgen/generational-arena
There are many, many arena implementations available with varying characteristics. It's disingenuous to act like Rust requires the author of an arena library to write "unsafe" everywhere.
I hadn't realized that Gogo was in such a bad spot with the upstream Go protobuf changes. There was lots of drama when the changes were made and I guess that overshadowed any optics I had on Gogo. Making vtprotobuf an additional protoc plugin seems like the Right Thing™, although it's a shame how complicated protoc commands end up becoming for mature projects. I'm pretty tempted to port Authzed over to this and run s…
Proto message unmarshal in Go for a small message should be 5 orders of magnitude below 20ms, shouldn't even begin to matter until you are sweating individual microseconds.
Perhaps they have significant external (network) latency leaving only a few ms budget for the application stack - so they could easily be up against a wall.
Using CPU utilization as a performance metric can be extremely misleading. My favorite article on the subject is from Brendan Gregg: http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-... A much better way to test the influence of the new compiler would be to test the actual throughput at which saturation is achieved (which is what the benchmark in the C++ grpc library measure to assess their performance).
Earlier quoted context omitted.
> That's what this whole thread is about: you can literally do just that I don't know how you get that from the thread: > Arenas are, however, unfeasible to implement in Go because it is a garbage collected language. > If you are willing to use cgo, google already implemented one for gapid. > there are other garbage collected languages like D, Nim and C# that offer the language features to do arenas without having to…
I was proposing the cgo option because it's already implemented. I _think_ allocating a slice of contiguous bytes and using unsafe pointers should work fine as long as you are very cautious about structs/vars with pointers into the buffer getting freed by the GC.
Go's GC is conservative, so I don't think you need to take any special caution in that regard. I would expect that you just need to take care that your casts are correct (e.g., that you aren't casting overlapping regions of memory as distinct objects).
Earlier quoted context omitted.
I can't believe we've managed to have this lengthy of a discussion about GC languages and speed without anyone mentioning rust. Has HN turned a corner?
Maybe, don't know. In what concerns me, although I like Rust, I only see it for scenarios where any kind of memory allocation is very precious, Ada/SPARK and MISRA-C style. I have been using GC languages with C++ like features, or polyglot codebases, for almost 20 years to think otherwise. Most of the time developers learn about new and miss out on the low level language features. It is a matter of balance, either tr…
I'm not sure that the phrasing in the article is particularly fair: > The maintainers of Gogo, understandably, were not up to the gigantic task. I'm 99% sure they are "up to" (as in "capable of") doing so, they are just not "up for" it (as in, "will not do it").
That said, I love the detailed post and the interesting solution, and the commitment to performance!
Earlier quoted context omitted.
I was proposing the cgo option because it's already implemented. I _think_ allocating a slice of contiguous bytes and using unsafe pointers should work fine as long as you are very cautious about structs/vars with pointers into the buffer getting freed by the GC.
> I _think_ allocating a slice of contiguous bytes and using unsafe pointers should work fine as long as you are very cautious about structs/vars with pointers into the buffer getting freed by the GC Go's GC is conservative, so I don't think you need to take any special caution in that regard. I would expect that you just need to take care that your casts are correct (e.g., that you aren't casting overlapping regions…
For example, it looks like pooled decoders could be implemented by setting a custom unmarshaller through the ProtoMethods[2] API.
I wonder why not? Did the authors of the vtprotobuf extension not want to bite off that much work? Is the new API not sufficient to do what they want (thus failing some of the goals expressed in golang/protobuf#364?
[1]: https://github.com/golang/protobuf/issues/364
[2]: https://pkg.go.dev/google.golang.org/protobuf@v1.26.0/reflec...