Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

Post reply on HN
Print view View original

221–230 of 250 posts

Re: Jemalloc Postmortem

#221

Post by vkazanov »

lloeki wrote: post #149
vkazanov wrote: post #129

Earlier quoted context omitted.

What prevents apple from working with gpl-style licenses is strict hatred towards code that they can't use without opensourcing it. So this is what prevents them from contributing to gpl projects: the need to control access to code. Llvm is OK for them from this point of view: upstream is open but they can maintain and distribute their proprietary fork.

> What prevents apple from working with gpl-style licenses is strict hatred towards code that they can't use without opensourcing it. Specifically regarding the C blocks feature introduced in Snow Leopard, as I recall, Apple wrote implementations for both clang and gcc, attempted to upstream the gcc patchset, said gcc patchset was obviously under a GPL license, but the GCC team threw a fit because it wanted the code…

Fsf is a non-profit organisation that proved numerous times that the point of its existence is making sure that I and we and you have freedom to change things we own.

I contribute and transfer copyright to them for my contributions for this sole reason.

Apple is not about freedoms at all.

That's OK. I mean, these are the meanings of what both orgs do. Understanding the system, I'd rather spend my free time on fsf cause (and make money in a commercial organisation).

Re: Jemalloc Postmortem

#222

Post by bbkane »

rstat1 wrote: post #190
ewalk153 wrote: post #135

Earlier quoted context omitted.

I’ve hit similar problems with their Ruby gRPC library. The counter example is the language Go. The team running Go has put considerable care and attention into making this project welcoming for developers to contribute, while still adhering to Google code contribution requirements. Building for source is straightforward and iirc it’s one of the easier cross compilers to setup. Install docs: https://go.dev/doc/instal…

Go is kinda of a pain to build from source. Build one version to build another, and another.. Or rather it was the last time I tried.

I think that's how most languages bootstrap.

Re: Jemalloc Postmortem

#223

Post by username223 »

einpoklum wrote: post #211
username223 wrote: post #206

Earlier quoted context omitted.

This. When step one is "install our weird build system," I'll immediately look for something else that meets my needs. All build systems suck, so everyone thinks they can write a better one, and too many people try. Pretty soon you end up having to learn a majority of this ( https://en.wikipedia.org/wiki/List_of_build_automation_softw... ) to get your code to compile.

If TCMalloc uses bazel, then you build it with Bazel. It just needs to install itself where you tell it to, and then either it has given you a pkg-config file, or otherwise, your own build system needs some library-finding logic for it ("find module" in CMake terms). Or - are you saying the problem is that you need to install Bazel?

> Or - are you saying the problem is that you need to install Bazel?

That. Then there's Facebook's build system, a few Python and JavaScript build systems, and pretty soon I have installed and have to deal with a half-dozen things that slightly improve upon Make. It's a maintenance burden if I ever have to touch any of these build systems.

Re: Jemalloc Postmortem

#224

Post by ahartmetz »

nh2 wrote: post #153
nh2 wrote: post #151

Earlier quoted context omitted.

The point of the blog post is that repo is over-focused on Facebook's needs instead of "general utility": > as a result of recent changes within Meta we no longer have anyone shepherding long-term jemalloc development with an eye toward general utility > we reached a sad end for jemalloc in the hands of Facebook/Meta > Meta’s needs stopped aligning well with those of external uses some time ago, and they are better o…

But I'd like to know exactly what that means. How can I find out if Facebook's focus is aligned with my own needs?

Now: Benchmarks :) Long term: you can look at the past and guess

Re: Jemalloc Postmortem

#225

Post by fc417fc802 »

mort96 wrote: post #170
fc417fc802 wrote: post #113

Earlier quoted context omitted.

Reading this perspective was interesting. I can appreciate that things didn't fit into your workflow very well, but my experience has been the opposite. Their projects seem to be structured from the perspective of building literally everything from source on the spot. That matches my mindset - I choose to build from scratch in a network isolated environment. As a result google repos are some of the few that I can cou…

I don't really have the care nor time to respond as thoroughly as you deserve, but here are some thoughts: > Out of curiosity which project did you run into this with? Their WebRTC library for the most part, but also the gRPC C++ library. Unlike WebRTC, grpc++ is in most package managers so the need to build it myself is less, but WebRTC is a behemoth and not in any package manager. > That said, isn't the only altern…

Yeah fair enough, controlling the build environment probably ought to be optional. Sounds like I dodged the issues you ran into due to the combination of specific library plus usecase. My experience is limited to abseil as well as the full dawn stack. In all cases I'm statically linking into my own applications, building everything except glibc & co from source, network isolated environment, using the same toolchain, compiler flags, etc.

Re: Jemalloc Postmortem

#226

Post by mkhnews »

cogman10 wrote: post #208
apaprocki wrote: post #75

Earlier quoted context omitted.

Ah, porting to HP Superdome servers. It’s like being handed a brochure describing the intricate details of the iceberg the ship you just boarded is about to hit in a few days. A fellow traveler, ahoy!

I worked on the Superdome servers back in the day. What a weird product. I still can't believe it was a profitable division (at my time circa 2011). HP was going through some turbulent waters in those days.

Yes, some good times despite all the work.

Re: Jemalloc Postmortem

#227

Post by JadedBlueEyes »

einpoklum wrote: post #211
username223 wrote: post #206

Earlier quoted context omitted.

This. When step one is "install our weird build system," I'll immediately look for something else that meets my needs. All build systems suck, so everyone thinks they can write a better one, and too many people try. Pretty soon you end up having to learn a majority of this ( https://en.wikipedia.org/wiki/List_of_build_automation_softw... ) to get your code to compile.

If TCMalloc uses bazel, then you build it with Bazel. It just needs to install itself where you tell it to, and then either it has given you a pkg-config file, or otherwise, your own build system needs some library-finding logic for it ("find module" in CMake terms). Or - are you saying the problem is that you need to install Bazel?

Building complete, optimised binaries can be much more complicated than just linking in a SO/A file. Things like cross-language LTO and PGO can be massive for performance and require integration throughout the build system.

Re: Jemalloc Postmortem

#228

Post by throwaway2037 »

kabdib wrote: post #210
matoro wrote: post #48

Earlier quoted context omitted.

That was me that filed the Itanium test suite failure. :)

one of the best books on Linux architecture i've read was the one on the Itanium port i think, because Itanic broke a ton of assumptions

This sounds interesting. Can you share the title of the book?

Re: Jemalloc Postmortem

#229

Post by sitkack »

cogman10 wrote: post #209
boulos wrote: post #181

Earlier quoted context omitted.

Oh, that wasn't the intent. I meant two separate things. The Itanic itself was kind of fascinating, but mostly panned (hence the nickname). SGI's decision to built out Itanium systems may have helped precipitate their own downfall. That was sad.

Still makes me sad. I partially think a major reason for the demise was that it was simply constructed too soon. Compiler tech wasn't nearly good enough to handle the ISA. Nowadays because of the efforts that have gone in to making SIMD effective, I'd think modern compilers would have an easier time taking advantage of that unique and strange uarch.

VLIW has a fatal flaw in how it was used in these systems. You cannot run general purpose dynamically scheduled workloads unless you combine the JIT engine and the scheduler. PRIOR ART. Which is the same exact problem of trying to run multiple compute kernels on a GPU at the same time. VLIW with an OS and runtime that uses a higher level language, Wasm or the JVM, could forseably support dynamic workloads where the main cpu was VLIW.

Now if they had been designed as GPU like devices for processing data, then Fortune 1000 would have never needed or used Hadoop.

Re: Jemalloc Postmortem

#230

Post by [deleted] »

jeffbee wrote: post #7
dpe82 wrote: post #5

Earlier quoted context omitted.

Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.

For an example of why an allocator is a maintenance treadmill, consider that C++ recently (relatively) added sized delete, and Linux recently gained transparent huge pages.

[deleted]
Post reply on HN