Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

131–140 of 250 posts

Re: Jemalloc Postmortem

#131
post #91

FTA: > And people find themselves in impossible situations where the main choices are 1) make poor decisions under extreme pressure, 2) comply under extreme pressure, or 3) get routed around. It doesn't sound like a work place :-(

Sounds like every workplace I've 'enjoyed' since ~2008

nice username

- fsociety

Re: Jemalloc Postmortem

#132

A bad choice of title, as "postmortem" made me think there was some severe outage caused by jemalloc.

The last part is unfortunate. However, it is a perfectly fine choice of title, as it does not make the majority of us think that there were an outage caused by jemalloc. You should update how you think of the word, and align it with the majority usage

Re: Jemalloc Postmortem

#133
post #108
post #105

Earlier quoted context omitted.

> they make no effort what so ever to distinguish between public header files and their source code They did, in a different way. The world is used to distinguish by convention, putting them in different directory hierarchy (src/, include/). google3 depends on the build system to do so, "which header file is public" is documented in BUILD files. You are then required to use their build system to grasp the difference…

> I have to disagree on this one. Relying on relative include paths suck. Just having one `-I/project/root` is the way to go. Oh to be clear, I'm not saying that they should've used relative includes. I'm complaining that they don't put their includes in their own namespace. If public headers were in a folder called `include/webrtc` as is the typical convention, and they all contained `#include ` or `#include "webrtc…

Thanks for the clarification. Yeah, that's pretty frustrating.

Now I'm curious why grpc, webrtc and some other Chromium repos were set up like this. Google projects which started in google3 and later exported as an open source project don't have this defect, for example tensorflow, abseil etc. They all had a top-level directory containing all their codes so it becomes `#include "tensorflow/...`.

Feels like a weird collision of coding style and starting a project outside of their monorepo

Re: Jemalloc Postmortem

#134
post #128

Earlier quoted context omitted.

I don't understand why you don't understand that you can be sad about this. Parent stated that he's sad the project is no longer maintained. That's a perfectly reasonable and human response. Parent does not have to defend having an emotion, even less provide objective truth for why he feels sad. If you don't agree, fine. But I don't see why one would write a paragraph long statement, I validating someone's emotional…

> That's a perfectly reasonable and human response. It's hardly a reasonable response. By casually saying that it's "sad" when a maintainer gives up on a project, GP is also including a very real and heavy implication that this is somehow wrong on their part and that they should continue to shoulder that burden, as a demand from the community. This is a really unhealthy attitude and we should all do away with it. Ins…

If my kid is sad that she cannot have an extra granola bar, that does not imply that I am wrong to deny her, or even that she thinks I am wrong, it just means she wishes she could have one.

Re: Jemalloc Postmortem

#135
post #98

Earlier quoted context omitted.

It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit o…

Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on w…

I’ve hit similar problems with their Ruby gRPC library.

The counter example is the language Go. The team running Go has put considerable care and attention into making this project welcoming for developers to contribute, while still adhering to Google code contribution requirements. Building for source is straightforward and iirc it’s one of the easier cross compilers to setup.

Install docs: https://go.dev/doc/install/source#bootstrapFromBinaryRelease

Re: Jemalloc Postmortem

#136

Earlier quoted context omitted.

> That's a perfectly reasonable and human response. It's hardly a reasonable response. By casually saying that it's "sad" when a maintainer gives up on a project, GP is also including a very real and heavy implication that this is somehow wrong on their part and that they should continue to shoulder that burden, as a demand from the community. This is a really unhealthy attitude and we should all do away with it. Ins…

If my kid is sad that she cannot have an extra granola bar, that does not imply that I am wrong to deny her, or even that she thinks I am wrong, it just means she wishes she could have one.

[flagged]

Re: Jemalloc Postmortem

#137

All the allocators have the same issue. They largely work against a shared set of allocation APIs. Many of their users mostly engage via malloc and free. So the flow is like this: user has an allocation looking issue. Picks up $allocator. If they have an $allocator type problem then they keep using it, otherwise they use something else. There are tons of users if these allocators but many rarely engage with the devel…

malloc is bad api in general, if you want to go fast you don't rely on general purpose allocator

Re: Jemalloc Postmortem

#138
post #91

FTA: > And people find themselves in impossible situations where the main choices are 1) make poor decisions under extreme pressure, 2) comply under extreme pressure, or 3) get routed around. It doesn't sound like a work place :-(

Sounds like every workplace I've 'enjoyed' since ~2008

Now I'm not one for victim blaming, but if that's more than three places of employment, maybe you need to rethink the positions you apply for.

Re: Jemalloc Postmortem

#139
Looking at all the comments and lightly browsing the source code, I'm amazed. Both at how much impact a memory allocator can make, but also how much code is involved.

I'm not really sure what I expected, but somehow I expect a memory allocator to be ... smaller, simpler perhaps?

Re: Jemalloc Postmortem

#140
post #20

Nice post -- so does Facebook no longer use jemalloc at all? Or is it maintenance mode? Or I wonder if they could simply use tcmalloc or another allocator these days? Facebook infrastructure engineering reduced investment in core technology, instead emphasizing return on investment.

Meta has a fork that they still are working on, where development is continuing. https://github.com/facebook/jemalloc

They take everything FLOSS and ruin it with bureaucracy, churn, breakage, and inconsideration to external use. They may claim FOSS broadly but it's mostly FOSS-washed, unusable garbage except for a few popular things.
Post reply on HN