Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

181–190 of 250 posts

Re: Jemalloc Postmortem

#181
post #97
post #54

Earlier quoted context omitted.

The Itanic was kind of great :). I'm convinced it helped sink SGI.

Why was the sinking of SGI great?

Oh, that wasn't the intent. I meant two separate things. The Itanic itself was kind of fascinating, but mostly panned (hence the nickname).

SGI's decision to built out Itanium systems may have helped precipitate their own downfall. That was sad.

Re: Jemalloc Postmortem

#182

Lesson: Don't let one megacorp dominate or take over your FOSS project. Push back somewhat and say "no" to too much help from one source.

He worked for like a decade at Facebook it looks like. I would guess at least at a Staff level. How many millions of dollars do you think he got from that? It doesnt sound like the worse trade in the world.

Re: Jemalloc Postmortem

#183
post #74

Your work was so impactful over a long period from Firefox to Facebook. Honored to have been a small part of it.

Your leadership on continuing investing in core technologies in Facebook were as fruitful as it could ever being. GraphQL, PyTorch, React to name a few cannot happen without.

Re: Jemalloc Postmortem

#184

Earlier quoted context omitted.

Why is that? Couldn’t there be push_simd()/pop_simd() that the syscall itself uses around its SIMD calls? If no syscalls use SIMD today, I’d think we’re starting from a safe position.

push_simd/pop_simd exist and are called kernel_fpu_begin/kernel_fpu_end. Their use is practically prohibited in most areas and iiuc not available on all archs, but it's available if needed.

Today I learned. Thanks!

Re: Jemalloc Postmortem

#185
post #166
post #149

Earlier quoted context omitted.

> What prevents apple from working with gpl-style licenses is strict hatred towards code that they can't use without opensourcing it. Specifically regarding the C blocks feature introduced in Snow Leopard, as I recall, Apple wrote implementations for both clang and gcc, attempted to upstream the gcc patchset, said gcc patchset was obviously under a GPL license, but the GCC team threw a fit because it wanted the code…

Big evil FSF, always trying to extract value and increase their stock price.

That was tongue-in-cheek, I thought Darth Vader's voice was enough of a cue.

License switches do happen though, and are the source of outrage. Cue redis.

The cause of transferring copyright is often practical (hard to track down + reach out to + gather answers from all authors-slash-contributors which hampers some critical decisions down the road); for the FSF it's ideological (GCC source code must remain under sole FSF control).

The consequence of the transfer though is not well understood by authors forfeiting their copyright: they essentially agree to work for free for whatever the codebase ends up being licensed to in the future, including possibly becoming entirely closed source.

Think of it next time you sign a CLA!

Re: Jemalloc Postmortem

#186
post #56

Earlier quoted context omitted.

It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit o…

I’ve successfully used LLMs to migrate Makefiles to bazel, more or less. I’ve not tried the reverse but suspect (2) isn’t so bad these days. YMMV, of course, but food for thought

Yep I've done something similar. This is the only way I managed to compile Google's C++ S2 library (spatial indexing) which depends on absl and OpenSSL.

(I managed to avoid infecting my project with boringSSL)

Re: Jemalloc Postmortem

#187

I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s…

> TCMalloc is great, but is an absolute nightmare to use if you’re not using bazel custom-malloc-newbie question: Why is the choice of build system (generator) significant when evaluating the usability of a library?

Because you have to build it. If they don't use the same build system as you, you either want to invoke their system, or import it into yours. The former is unappealing if it's 'heavy' or doesn't play well as a subprocess; the latter can take a lot of time if the build process you're replicating is complex.

I've done both before, and seen libraries at various levels of complexity; there is definitely a point where you just want to give up and not use the thing when it's very complex.

Re: Jemalloc Postmortem

#188

I've used jemalloc in every game engine I've written for years. It's just the thing to do. WAY faster on win32 than the default allocator. It's also nice to have the same allocator across all platforms. I learned of it from it's integration in FreeBSD and never looked back. jemalloc has help entertained a lot of people :)

+1

windows def allocator is pos. Jemalloc rules

Re: Jemalloc Postmortem

#189
post #98

Earlier quoted context omitted.

It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit o…

Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on w…

I agree to a point. grpc++ (and protobuf and boringssl and abseil and....) was the biggest pain in the ass to integrate in to a personal project I've ever seen. I ended up having to write a custom tool to convert their Bazel files to the format my projects tend to use (GN and Ninja). Many hours wasted. There were no library specfici "sysroots" or "toolchains" involved though thankfully because I'm sure that would made things even worse.

Upside is (I guess) if I ever want to use grpc in another project the work's already done and it'll just be a matter of copy/paste.

Re: Jemalloc Postmortem

#190
post #98

Earlier quoted context omitted.

Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on w…

I’ve hit similar problems with their Ruby gRPC library. The counter example is the language Go. The team running Go has put considerable care and attention into making this project welcoming for developers to contribute, while still adhering to Google code contribution requirements. Building for source is straightforward and iirc it’s one of the easier cross compilers to setup. Install docs: https://go.dev/doc/instal…

Go is kinda of a pain to build from source. Build one version to build another, and another..

Or rather it was the last time I tried.

Post reply on HN