Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

Post reply on HN
Print view View original

161–170 of 250 posts

Re: Jemalloc Postmortem

#161

Post by acdha »

mrweasel wrote: post #138
bravetraveler wrote: post #101

Earlier quoted context omitted.

Sounds like every workplace I've 'enjoyed' since ~2008

Now I'm not one for victim blaming, but if that's more than three places of employment, maybe you need to rethink the positions you apply for.

There’s something to that but it is victim blaming if you’re not acknowledging the larger trends. There are a lot of places whose MBAs are attending the same conferences, getting the same recommendations from consultants, and hearing the same demands from investors. The push against remote work, for example, was all driven by ideology against most of the available data but it affected a huge number of jobs.

Re: Jemalloc Postmortem

#162

Post by KingLancelot »

adgjlsfhk1 wrote: post #29
meisel wrote: post #19

I believe there’s no other allocator besides jemalloc that can seamlessly override macOS malloc/free like people do with LD_PRELOAD on Linux (at least as of ~2020). jemalloc has a very nice zone-based way of making itself the default, and manages to accommodate Apple’s odd requirements for an allocator that have tripped other third-party allocators up when trying to override malloc/free.

I believe mimalloc works here (but might be wrong).

[dead]

Re: Jemalloc Postmortem

#163

Post by durrrrrrrrrrrrr »

LtdJorge wrote: post #121
senderista wrote: post #86

Earlier quoted context omitted.

> Semi-related: one thing that most people never think about: it is exactly the same amount of work for the kernel to zero a page of memory (in preparation for a future mmap) as for a userland process to zero it out (for its own internal reuse) Possibly more work since the kernel can't use SIMD

Why is that? Doesn't Linux use SIMD for the crypto operations?

It's not so much that you can't ever use it, it's more a you really shouldn't. It's more expensive, harder to use and rarely worth it. Main users currently are crypto and raid checksumming.

https://www.kernel.org/doc/html/next/core-api/floating-point...

Re: Jemalloc Postmortem

#164

Post by kstrauser »

dwattttt wrote: post #150
LtdJorge wrote: post #121

Earlier quoted context omitted.

Why is that? Doesn't Linux use SIMD for the crypto operations?

Allowing SIMD instructions to be used arbitrarily in kernel actually has a fair penalty to it. I'm not sure what Linux does specifically, but: When a syscall is made, the kernel has to backup the user mode state of the thread, so it can restore it later. If any kernel code could use SIMD registers, you'll have to backup and restore that too, and those registers get big. You could easily be looking at adding a 1kb cop…

Why is that? Couldn’t there be push_simd()/pop_simd() that the syscall itself uses around its SIMD calls?

If no syscalls use SIMD today, I’d think we’re starting from a safe position.

Re: Jemalloc Postmortem

#165

Post by throw0101d »

acdha wrote: post #161
mrweasel wrote: post #138

Earlier quoted context omitted.

Now I'm not one for victim blaming, but if that's more than three places of employment, maybe you need to rethink the positions you apply for.

There’s something to that but it is victim blaming if you’re not acknowledging the larger trends. There are a lot of places whose MBAs are attending the same conferences, getting the same recommendations from consultants, and hearing the same demands from investors. The push against remote work, for example, was all driven by ideology against most of the available data but it affected a huge number of jobs.

> The push against remote work, for example, was all driven by ideology against most of the available data but it affected a huge number of jobs.

And before that, open office plans.

You're saving on rent: great. But what is it doing to productivity?

* https://business.adobe.com/blog/perspectives/what-science-sa...

Of course productivity doesn't show up on a spreadsheet, but rent does, so it's what about "the numbers" say.

Re: Jemalloc Postmortem

#166

Post by Y_Y »

lloeki wrote: post #149
vkazanov wrote: post #129

Earlier quoted context omitted.

What prevents apple from working with gpl-style licenses is strict hatred towards code that they can't use without opensourcing it. So this is what prevents them from contributing to gpl projects: the need to control access to code. Llvm is OK for them from this point of view: upstream is open but they can maintain and distribute their proprietary fork.

> What prevents apple from working with gpl-style licenses is strict hatred towards code that they can't use without opensourcing it. Specifically regarding the C blocks feature introduced in Snow Leopard, as I recall, Apple wrote implementations for both clang and gcc, attempted to upstream the gcc patchset, said gcc patchset was obviously under a GPL license, but the GCC team threw a fit because it wanted the code…

Big evil FSF, always trying to extract value and increase their stock price.

Re: Jemalloc Postmortem

#167

Post by didip »

Thanks for everything, JE!

jemalloc is always the first thing I installed whenever I had to provision bare servers.

If jemalloc is somehow the default allocator in Linux, I think it will not have a hard time retaining contributors.

Re: Jemalloc Postmortem

#168

Post by pvg »

zozbot234 wrote: post #130
7bit wrote: post #128

Earlier quoted context omitted.

I don't understand why you don't understand that you can be sad about this. Parent stated that he's sad the project is no longer maintained. That's a perfectly reasonable and human response. Parent does not have to defend having an emotion, even less provide objective truth for why he feels sad. If you don't agree, fine. But I don't see why one would write a paragraph long statement, I validating someone's emotional…

> That's a perfectly reasonable and human response. It's hardly a reasonable response. By casually saying that it's "sad" when a maintainer gives up on a project, GP is also including a very real and heavy implication that this is somehow wrong on their part and that they should continue to shoulder that burden, as a demand from the community. This is a really unhealthy attitude and we should all do away with it. Ins…

You don't really get to decide whether someone's entirely in-passing mention of an emotional response to developments in something they worked on is 'reasonable' or not, and more importantly, it's not at all a topic of interesting conversation, just pedantic nitpicking.

Re: Jemalloc Postmortem

#169

Post by adityapatadia »

laszlojamf wrote: post #92
adityapatadia wrote: post #84

Jason, here is a story about how much your work impacts us. We run a decently sized company that processes hundreds of millions of images/videos per day. When we first started about 5 years ago, we spent countless hours debugging issues related to memory fragmentation. One fine day, we discovered Jemalloc and put it in our service, which was causing a lot of memory fragmentation. We did not think that those 2 lines o…

I really don't mean to be snarky, but honest question: Did you donate? Nothing says thank you like some $$$...

We regularly donate to project via open collective. We frankly did not see here due to FB involvement I think.

Re: Jemalloc Postmortem

#170

Post by mort96 »

fc417fc802 wrote: post #113
mort96 wrote: post #98

Earlier quoted context omitted.

Everything from Google is an absolute pain to work with unless you're in Google using their systems, FWIW. Anything from the Chromium project is deeply intangled with everything else from the Chromium project as part of one gigantic Chromium source tree with all dependencies and toolchains vendored. They do not care about ABI what so ever, to the point that a lot of Google libraries change their public ABI based on w…

Reading this perspective was interesting. I can appreciate that things didn't fit into your workflow very well, but my experience has been the opposite. Their projects seem to be structured from the perspective of building literally everything from source on the spot. That matches my mindset - I choose to build from scratch in a network isolated environment. As a result google repos are some of the few that I can cou…

I don't really have the care nor time to respond as thoroughly as you deserve, but here are some thoughts:

> Out of curiosity which project did you run into this with?

Their WebRTC library for the most part, but also the gRPC C++ library. Unlike WebRTC, grpc++ is in most package managers so the need to build it myself is less, but WebRTC is a behemoth and not in any package manager.

> That said, isn't the only alternative for them moving to something like nix? Otherwise how do you tightly specify the build environment?

I don't expect my libraries to tightly specify the build environment. I expect my libraries to conform to my software's build environment, to use versions of other libraries that I provide to it, etc etc. I don't mind that Google builds their application software the way they do, Google Chrome should tightly constrain its build environment if Google wants; but their libraries should fit in to my environment.

I'm wondering, what is your relationship with Google software that you build from source? Are you building their libraries to integrate with your own applications, or do you just build Google's applications from source and use them as-is?

Post reply on HN