Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debugging a memory leak in a Clojure service

charanvasu.com

Post reply on HN
Print view View original

21–22 of 22 posts

Re: Debugging a memory leak in a Clojure service

#21

Post by ysleepy »

henning wrote: post #5

If you can go from ~60ms p99 response times to ~45 from reduced garbage collection, that means GC has a major impact on user-perceptible performance on your application and proves that it is an extremely expensive operation that should be carefully managed. If you have a modern microservices Kubernetes blah blah bullshit setup, this fraud detection service is probably only one part of a chain of service calls that oc…

They only guess the performance difference is because of GC, generating code on the fly and compiling it to classes in your hot path is also probably not cheap.

Memory allocation/deallocation overhead is always present, just look at different allocators, fragmentation issues and so on. Using a GC is not intrinsically much different performance wise.

Re: Debugging a memory leak in a Clojure service

#22

Post by NightMKoder »

positr0n wrote: post #16
NightMKoder wrote: post #11

If your clojure pods are getting OOMKilled, you have a misconfigured JVM. The code (e.g. eval or not) mostly doesn't matter. If you have an actual memory leak in a JVM app what you want is an exception called java.lang.OutOfMemoryError . This means the heap is full and has no space for new objects even after a GC run. An OOMKilled means the JVM attempted to allocate memory from the OS but the OS doesn't have any memo…

> It never gets a chance to say "man I should clear up some space cause I'm running out". To add to everything you said, depending on the type of framework you are using sometimes you don't even want it to do that. The JVM will try increasingly desperate measures, looped GC scans, ref processing, and sleeps with backoffs. With a huge heap, that can easily take hundreds to thousands of ms. At scale, it's often better…

Of course you’re right and you really want to avoid getting to GC thrashing. IMO people still miss the old +UseGCOverheadLimit on the new GCs.

That said trying to enforce overhead limits with RSS limits also won’t end well. Java doesn’t make guarantees around max allocated but unused heap space. You need something like this: https://github.com/bazelbuild/bazel/blob/10060cd638027975480... - but I have rarely seen something like that in production.

Post reply on HN