151–160 of 259 posts
Earlier quoted context omitted.
> By the time you joined and benchmarked these systems, the continuous rolling deployment had taken over Nope, I started in 2014. > I don't recall ever talking to you on the matter. I recall. You refused to believe the benchmark results and made me repeat the test, then stopped replying after I did :)
This is why I love hacker news. I learn so much from these moments.
Earlier quoted context omitted.
For the peanut gallery more: I worked with both of these guys at Meta on this. The "servers are only on for a few hours" thing was like never true so I have no idea where that claim is coming from. The web performance test took more than a few hours to run alone and we had way more aggressive soaks for other workloads. My recollection was that "write zeroes" just became a cheaper operation between '12 and '14. A fun…
[ Edit: "servers" in this context meant the HHVM server processes, not the physical server which of course had a longer uptime ] People got promoted for continuous deployment https://engineering.fb.com/2017/08/31/web/rapid-release-at-m... I think it's fair to say the hardware changed, the deployment strategy changed and the patches were no longer relevant, so we stopped applying them. When I showed up, there were 100…
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.
As an Australian who was just made redundant from a role that involved this type of low level programming - I love working on these these kinds of challenges. I'm saddened that the job market in Australia is largely React CRUD applications and that it's unlikely I will find a role that lets me leverage my niche skill set (which is also my hobby)
Earlier quoted context omitted.
Reinventing data structures poorly is very common. Heap allocation in java is something trivial happens constantly. People typically do funky stuff with memory allocation because they have to, because the GC is causing pauses. People avoid system allocators in C++ too, they just don't have to do it because of uncontrollable pauses.
> People typically do funky stuff with memory allocation because they have to This same dev did things like putting what he deemed as being large objects (icons) into weak references to save memory. When the references were collected, invariably they had to be reloaded. That was not the source of memory pressure issues in the app. I've developed a mistrust for a lot of devs "doing it because we have to" when it comes…
Any program that cares about performance is going to focus on minimizing memory allocation first. The difference between a GCed language like java is that the problems manifest as gc pauses that may or may not be predictable. In a language like C++ you can skip the pauses and worry about the overall throughput.
Earlier quoted context omitted.
This is why I love hacker news. I learn so much from these moments.
Like "never work at Meta unless you can out-toxic your coworkers".
Earlier quoted context omitted.
Then there’s perl, which doesn’t free at all.
doesn't java also? I heard that was a common complaint for minecraft
Earlier quoted context omitted.
If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).
Agreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc.
Earlier quoted context omitted.
> Facebook's coding AIs to the rescue, maybe? I wonder how good all these "agentic" AIs are at dreaded refactoring jobs like these. No. This is something you shouldn't allow coding agents anywhere near, unless you have expert-level understanding required to maintain the project like the previous authors have done without an AI for years.
Hm, I wonder. I've done some work in this sort of area before, though not literally on a malloc. Yes you very much want to be careful, but ultimately it's the tests that give you confidence. Pound the heck out of it in multithreaded contexts and test for consistency.
Earlier quoted context omitted.
We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like…
I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator). Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allo…
Indeed, it's not the old gperftools version.
Blog: https://abseil.io/blog/20200212-tcmalloc
History / Diffs: https://google.github.io/tcmalloc/gperftools.html
Earlier quoted context omitted.
Like "never work at Meta unless you can out-toxic your coworkers".
Yea I knew meta was toxic, but publicly beefing over something over a decade ago is a whole other matter. I can’t even remember what I was working on 10 years ago, and even if I did I wouldn’t be bringing people down that much later.