Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta’s renewed commitment to jemalloc

engineering.fb.com

Post reply on HN
Print view View original

121–130 of 259 posts

Re: Meta’s renewed commitment to jemalloc

#121

Post by ajxs »

apatheticonion wrote: post #117

As an Australian who was just made redundant from a role that involved this type of low level programming - I love working on these these kinds of challenges. I'm saddened that the job market in Australia is largely React CRUD applications and that it's unlikely I will find a role that lets me leverage my niche skill set (which is also my hobby)

Speaking as an Australian that works on React CRUD applications because there's nothing else in the market, I've been reading through this thread thinking the exact same thing.

Re: Meta’s renewed commitment to jemalloc

#122

Post by cogman10 »

CyberDildonics wrote: post #118
cogman10 wrote: post #106

Earlier quoted context omitted.

People do stuff they shouldn't all the time. For example, some code I had to clean up pretty early on in my career was a dev, for unknown reasons, reinventing the `ArrayList` and then using that invention as a set (doing deduplication by iterating over the elements and checking for duplicates). It was done in the name of performance, but it was never a slow part of the code. I replaced the whole thing with a `HashSet…

Reinventing data structures poorly is very common. Heap allocation in java is something trivial happens constantly. People typically do funky stuff with memory allocation because they have to, because the GC is causing pauses. People avoid system allocators in C++ too, they just don't have to do it because of uncontrollable pauses.

> People typically do funky stuff with memory allocation because they have to

This same dev did things like putting what he deemed as being large objects (icons) into weak references to save memory. When the references were collected, invariably they had to be reloaded.

That was not the source of memory pressure issues in the app.

I've developed a mistrust for a lot of devs "doing it because we have to" when it comes to performance tweaks. It's not a never thing that a buffer is the right thing to do, but it's not been something I had to reach for to solve GC pressure issues. Often times, far more simple solutions like pulling an allocation out of the middle of a loop, or switching from boxed types to primatives, was all that was needed to relieve memory pressure.

The closest I've come to it is replacing code which would do an expensive and allocation heavy calculation with a field that caches the result of that calculation on the first call.

Re: Meta’s renewed commitment to jemalloc

#123

Post by jeffbee »

Dylan16807 wrote: post #103
jeffbee wrote: post #67

Earlier quoted context omitted.

Right but on Intel the 1G page size has historically been the odd one. For example Skylake-X has 1536 L2 shared TLB entries for either 4K or 2M pages, but it only has 16 entries that can be used for 1G pages. It wasn't unified until Cascade Lake. But Skylake-like Xeon is still incredibly common in the cloud so it's hard to target the later ones.

So for any process that's using less than 16GB, it's a significant performance boost. And most processes using more RAM, but not splitting accesses across more than 16 zones in rapid succession, will also see a performance boost. My old Intel CPU only has 4 slots for 1GB pages, and that was enough to get me about a 20% performance boost on Factorio. (I think a couple percent might have been allocator change but the b…

That strikes me as a common hugepages win. People never believe you, though, when you say you can make their thing 20% faster for free.

Re: Meta’s renewed commitment to jemalloc

#124

Post by drob518 »

cogman10 wrote: post #75
bluGill wrote: post #50

Earlier quoted context omitted.

When it works. Many programs in GC language end up fighting the GC by allocating a large buffer and managing it by hand anyway because when performance counts you can't have allocation time in there at all. (you see this in C all the time as well)

That's generally a bad idea. Not always, but generally. It was a better idea when Java had the old mark and sweep collector. However, with the generational collectors (which are all Java collectors now. except for epsilon) it's more problematic. Reusing buffers and objects in those buffers will pretty much guarantees that buffer ends up in oldgen. That means to clear it out, the VM has to do more expensive collection…

If your workload is very regular, you can still do better with an arena allocator. Within the arena, it uses the same pointer-bump allocation as Java normally uses, but then you can free the whole area back to the start by resetting the pointer to its initial value. If you use the arena for servicing a single request, for instance, you then reset as soon as you're done with the request, setting you up with a totally free area for the next request. That's more efficient than a GC. But it also requires your algorithm to fall into that pattern where you KNOW that you can and should throw everything from the request away. If you can't guarantee that, then modern collectors are pretty magical and tunable.

Re: Meta’s renewed commitment to jemalloc

#125

Post by snackbroken »

adgjlsfhk1 wrote: post #91
cogman10 wrote: post #69

Earlier quoted context omitted.

Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.

This only really ends up being a problem on windows. On systems with proper virtual memory setups, the cost of unused memory is very low (since the the OS can just page it out)

For video games it is pretty bad, because reading back a page from disk containing "freed" (from the application perspective, but not returned to the OS) junk you don't care about is significantly slower than the OS just handing you a fresh one. A 10-20ms delay is a noticeable stutter and even on an SSD that's only a handful of round-trips.

Re: Meta’s renewed commitment to jemalloc

#126

Post by Pepe1vo »

apatheticonion wrote: post #117

As an Australian who was just made redundant from a role that involved this type of low level programming - I love working on these these kinds of challenges. I'm saddened that the job market in Australia is largely React CRUD applications and that it's unlikely I will find a role that lets me leverage my niche skill set (which is also my hobby)

Not sure if it's the domain you're interested in, but there are quite a few HFT firms with offices in Australia.

The one I know of (IMC trading) does a lot of low level stuff like this and is currently hiring.

Re: Meta’s renewed commitment to jemalloc

#128

Post by drob518 »

cogman10 wrote: post #122
CyberDildonics wrote: post #118

Earlier quoted context omitted.

Reinventing data structures poorly is very common. Heap allocation in java is something trivial happens constantly. People typically do funky stuff with memory allocation because they have to, because the GC is causing pauses. People avoid system allocators in C++ too, they just don't have to do it because of uncontrollable pauses.

> People typically do funky stuff with memory allocation because they have to This same dev did things like putting what he deemed as being large objects (icons) into weak references to save memory. When the references were collected, invariably they had to be reloaded. That was not the source of memory pressure issues in the app. I've developed a mistrust for a lot of devs "doing it because we have to" when it comes…

Premature optimization is the root of all evil.

Re: Meta’s renewed commitment to jemalloc

#129

Post by apatheticonion »

Pepe1vo wrote: post #126
apatheticonion wrote: post #117

As an Australian who was just made redundant from a role that involved this type of low level programming - I love working on these these kinds of challenges. I'm saddened that the job market in Australia is largely React CRUD applications and that it's unlikely I will find a role that lets me leverage my niche skill set (which is also my hobby)

Not sure if it's the domain you're interested in, but there are quite a few HFT firms with offices in Australia. The one I know of (IMC trading) does a lot of low level stuff like this and is currently hiring.

I'm actually looking at HFT companies. Hoping I find one that allows remote working - but looks like there are basically no remote roles going at the moment

Re: Meta’s renewed commitment to jemalloc

#130

Post by cogman10 »

adgjlsfhk1 wrote: post #91
cogman10 wrote: post #69

Earlier quoted context omitted.

Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.

This only really ends up being a problem on windows. On systems with proper virtual memory setups, the cost of unused memory is very low (since the the OS can just page it out)

Unfortunately, the JVM and collectors like the JVM's plays really bad with virtual memory. (Actually, G1 might play better. Everything else does not).

The issue is that through the standard course of a JVM application running, every allocated page will ultimately be touched. The JVM fills up new gen, runs a minor collection, moves old objects to old gen, and continues until old gen gets filled. When old gen is filled, a major collection is triggered and all the live objects get moved around in memory.

This natural action of the JVM means you'll see a sawtooth of used memory in a properly running JVM where the peak of the sawtooth occasionally hits the memory maximum, which in turn causes the used memory to plummet.

Post reply on HN