Live data from Hacker News

The Magic of strace

chadfowler.com

21–30 of 105 posts

Re: The Magic of strace

#21
Mac OS X has a suite of tools built on a similar package called dtrace—opensnoop and execsnoop. Gives really nice real time lists of all files opened on the system and all binaries executed, respectively.

Re: The Magic of strace

#22
post #14

Thanks for the writeup! strace should definitely be in your toolbox. There is also systemtap, which I like a lot as well. It has some problem on Linux though, especially only being widely supported of Linux > 3.5 if the distro you are using does not ship with patches. Custom userspace probes are a real strong point. I wrote a short article about stap using Rubys probes as an example: http://www.asquera.de/blog/2014-0…

To clarify, systemtap needs UTRACE or UPROBES kernel support to trace user processes. Without those you can still inspect kernel functions. IIRC UTRACE was mainly supported by Redhat, and available intheir kernels for quite a few years. In 3.5 the UPROBES functionality was merged in to mainline. Other tools, like perf, can use UPROBES support as well.

Re: The Magic of strace

#25
"perf top -e syscalls:statfs"

particularly when you don't know which process is calling all the syscalls.

Mix "perf record" and "perf trace" & you have the next generation of strace tools.

Re: The Magic of strace

#26
post #14

Thanks for the writeup! strace should definitely be in your toolbox. There is also systemtap, which I like a lot as well. It has some problem on Linux though, especially only being widely supported of Linux > 3.5 if the distro you are using does not ship with patches. Custom userspace probes are a real strong point. I wrote a short article about stap using Rubys probes as an example: http://www.asquera.de/blog/2014-0…

To clarify, systemtap needs UTRACE or UPROBES kernel support to trace user processes. Without those you can still inspect kernel functions. IIRC UTRACE was mainly supported by Redhat, and available intheir kernels for quite a few years. In 3.5 the UPROBES functionality was merged in to mainline. Other tools, like perf, can use UPROBES support as well.

Yep, thats the detailed story. A lot of Linuxes (especially current Ubuntu LTS) ship without those patches, though, which makes the whole exercise of compiling the kernel yourself (and maybe a debug image to go along with it) a tedious one, and probably not fit for a production environment.

Thats why I decided to focus on Linux > 3.5, which will be available with Ubuntu 14.04 LTS, where installing gets much easier if you know the right packages. I definitely wanted to make sure that people can start playing around with it in a few minutes.

Also, UPROBES are in my opinion one of the most interesting features to grab with strace. It allows you to easily combine detailed kernel-level tracing with tracing of your application.

(Not that I want to suggest that compiling your own kernel is hard to do, it just takes the fun out of "let's trace!")

Re: The Magic of strace

#28

Please see ftp://86.0.252.89/pub/release/website/tools/trace-20140126-x86_64-b95.tar.gz This is a tool called ptrace - which does everything that strace does and a lot more. You have working binaries in there, and most of the source - I havent extricated the full build dependencies so it all builds, but this includes extra facilities like reporting summaries of process trees, showing only connections or files, and sh…

I am a bit paranoid about downloading a binary from a site without a domain name.

I can understand being twitchy but it is so easy to get a domain name that it would literally be no barrier at all to an attacker. Not wanting a binary (especially one that will run with root privs) to come from a domain you don't recognise and trust is understandable. Even then it should be by https with AND/OR a signed package are the first steps to security.

Re: The Magic of strace

#29
post #20

If you think strace is useful, wait until you try dtrace.

dtrace is cool, and it benefits from source instrumentation (probes).

strace / dtruss just spit out kernel syscalls of any user land program wo modifications.

Also dtrace, as seen on smartos

Re: The Magic of strace

#30
Just a few hours ago, a newly minted Ubuntu binary was crashing due a library version mismatch. I thought I had updated the shared libraries to point to the new versions. But definitely something was still hooked to the old version. I just couldn't figure out how/where. ldd wasn't of much help because everything was OK according to it. "If only I can get a bit more info when the binary is running and spit out everything before the crash."

Tried my luck with gdb. Sure enough...there was libQt5DBus pointing to the old libs leading to the crash. If you are feeling particularly adventurous, you can step one instruction at a time after starting. Even without debug symbols, there is quite a lot of info that be used while troubleshooting.

Post reply on HN