Could you compare/contrast its functionality to https://github.com/KDE/heaptrack ?
Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
11–20 of 31 posts
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#12So it's like strace looking for brk()?
Sorry if this is a dumb question, but can't strace trace brk() calls? And as kind of a follow up what is the easiest way to trace all allocations (brk() and mmap) but nothing else?
> can't strace trace brk() calls?
Absolutely.
> what is the easiest way to trace all allocations (brk() and mmap) but nothing else?
strace -e mmap "$command"
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#13Interesting approach. How is performance compared to something like https://github.com/koute/bytehound
Bytehound author here. Just from a cursory look at the README: > allocscope-trace attaches to another process as a debugger. By using breakpoints on memory allocation functions such as malloc it tracks allocations made by that process. Looks like it's using breakpoints so I'd expect it to be orders of magnitude slower. And looking at the source code it's also using `libunwind`, so even if it wasn't using breakpoints…
Out of curiosity I ran a quick test on my private benchmark.
libbytehound.so (with extra debug assertions, because I'm too lazy to recompile in release mode): 4s
allocscope: did not finish after 4 minutes (I got bored waiting and CTRL+C'd it)
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#14So it's like strace looking for brk()?
strace is limited to system calls but this particular tool uses ptrace to trap symbolic references to mmap, malloc, calloc, etc. This provides better resolution because your allocator probably asks for large chunks of memory from the system and allocates from those instead of making each request one-for-one.
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#15Neat. I had made something similar for work a while back, but as a LD_PRELOAD library that intercepted calls to malloc and friends. It would add extra space to every allocation so it could add a pointer at the end that would point into a leaf node of a call graph backtrace tree it maintained. Each node in the tree had lifetime allocated/freed block counts and bytes by code site. The cool part about it was that it barely affected the performance of the application.
It made its own socket and thread to listen on it. It would just dump a snapshot of tree to anything that connected. I also had some tooling that would let you diff two snapshots, since it was helpful to see if particular stimuli cause persistent extra allocations. While finding the largest outstanding delta between allocated and free bytes was great for finding leaks, sorting by lifetime count of blocks allocated was also fun. I remember some little puzzle game I enjoyed playing at the time would allocate and free tens of thousands of blocks as you dragged a line around for a second.
There was a tricky chicken and egg problems with LD_PRELOAD wrapping one of the allocation functions, because it was used internally by dlsym, which I was using to retrieve pointers to the proper function implementations. (calloc if I recall correctly.) I hacked around it by making my library allocate bytes out of a static char array for the calloc call that would happen while dlsym-ing for calloc. Debugging this was a nightmare, since it would break so early in the process's lifetime that GDB breakpoints weren't functioning. Tracking in a second process seems like a way simpler idea, and probably doesn't have too much of an impact on performance.
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#16I really like the picture at the top, was that the work of stable diffusion?
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#17If this project can trace memory allocation/deallocation and their call stacks in real time -- this would be super useful, because we can statistically profile which function is always allocating without proper free in a certain time frame (when the memory is supposed to be freed), because valgrind only tells you there are memory leaks but not where is the leak exactly.
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#18I'd be curious to see how this ptrace tool performs compared with one that relies on ELF symbol interposition (a la LD_PRELOAD). Other heap profilers (heaptrack, libtcmalloc, etc) use this method. Presumably the loader resolves the symbols once at load time and there's little cost overhead to switch to the profiler code.
However, as a practical matter those solutions might omit mmap which some applications might use for anonymous allocations.
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#19I'd be curious to see how this ptrace tool performs compared with one that relies on ELF symbol interposition (a la LD_PRELOAD). Other heap profilers (heaptrack, libtcmalloc, etc) use this method. Presumably the loader resolves the symbols once at load time and there's little cost overhead to switch to the profiler code. However, as a practical matter those solutions might omit mmap which some applications might use…
> I'd be curious to see how this ptrace tool performs compared with one that relies on ELF symbol interposition (a la LD_PRELOAD).
I've posted some very quick numbers in my comment here comparing it to Bytehound: https://news.ycombinator.com/item?id=34806401
> However, as a practical matter those solutions might omit mmap which some applications might use for anonymous allocations.
Bytehound also gathers mmaps. (:
Re: Show HN: I wrote a tool in Rust for tracking all allocations in a Linux process
#20Assuming you're running linux, there are some ebpf programs that can accomplish this already, no breakpoints needed.