Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Magic of strace

chadfowler.com

Post reply on HN
Print view View original

71–80 of 105 posts

Re: The Magic of strace

#71

Post by eldavido »

I use strace all the time doing ops at Crittercism. Some of the random things it's helped with/taught me:

- allowed exploring forking behavior of daemons, in particular the nitty-gritty of gunicorn's prefork behavior, and understanding the rationale behind single- and double-fork daemons generally (very important to understand for job control e.g. writing upstart/init.d jobs)

- isolated hot reads to memcache in situ, by identifying the socket associated with the memcache connection, and finding which key was read the most by a process (we built better logging after the fact, but sometimes there's no substitute for instrumenting prod during tough perf/stress problems)

- let me explore the behavior of node.js's several threads, and find one of them sending "X" over a socket to the other (still not quite sure what this is, some kind of heartbeat/clock tick?)

- helped understanding "primordial processes" and the exact details of how forking/reparenting work on linux

It's a great tool and one that every ops/infrastructure engineer should be familiar with.

Re: The Magic of strace

#72

Post by mrfusion »

peterwwillis wrote: post #68
mrfusion wrote: post #50

Has anyone heard of a program that will take strace (or dtrace) output and create a pretty diagram showing which commands call which commands and which files they read or create? We've got a fairly complicated bioinformatics pipeline that calls about 100 other programs, and creates or reads about 100 different files. I'd love a way to create a picture of what's going on. Which files each program uses, etc. If such a…

Sounded like an interesting script, so I just wrote it in about a half hour. You're welcome to sell it if you want... (also, there's probably a default recursion limit of 100; unroll the recursion in walkpid to go farther) Collect logs with 'strace -o pids.log -e trace=process -f [specify your process here]', run with 'perl printpids.pl #!/usr/bin/perl -w $|=1; use strict; my (%pidmap, @order); while ( ) { chomp; if…

Wow that's amazing! Can I have strace launch the program? Or does it already have to be running?

Re: The Magic of strace

#75

Post by alexnewman »

eldavido wrote: post #71

I use strace all the time doing ops at Crittercism. Some of the random things it's helped with/taught me: - allowed exploring forking behavior of daemons, in particular the nitty-gritty of gunicorn's prefork behavior, and understanding the rationale behind single- and double-fork daemons generally (very important to understand for job control e.g. writing upstart/init.d jobs) - isolated hot reads to memcache in situ,…

It's still no dtrace. Something that I hope enters osx soon.

Re: The Magic of strace

#76

Post by dunham »

Xdes wrote: post #63
dmpk2k wrote: post #58

Earlier quoted context omitted.

strace shows syscalls -- it's effectively truss. That's useful and all, but what if you want to instrument arbitrary parts of a program, not just the syscall interface? By function or instruction? Either in userspace or in kernel? With statistical functions? And speculative tracing? And extensive control flow (except loops, which prevent certain safely guarantees DTrace makes). And a lot more. Don't be fooled by the…

Still doesn't tell me what dtrace does.

It's like awk, except that you match entry/exit of syscalls, function calls, method invocations (in ObjC/Java), and give code to execute with access to arguments, return values, stack trace, etc.

It can be used to write tools like strace (see "dtruss" on OSX), iotop, topsyscall, etc.

Re: The Magic of strace

#78

Post by dicroce »

Also check out ltrace... Shows the calls to other libraries the process is making...

I'd also like to point out that a key to using strace successfully is the result column... Programs that fail often make system calls that fail right before they exit... You can often tell what the program is trying and failing to accomplish...

Re: The Magic of strace

#79

Post by retr0h »

alexnewman wrote: post #75
eldavido wrote: post #71

I use strace all the time doing ops at Crittercism. Some of the random things it's helped with/taught me: - allowed exploring forking behavior of daemons, in particular the nitty-gritty of gunicorn's prefork behavior, and understanding the rationale behind single- and double-fork daemons generally (very important to understand for job control e.g. writing upstart/init.d jobs) - isolated hot reads to memcache in situ,…

It's still no dtrace. Something that I hope enters osx soon.

dtrace has been in osx since 10.5 :/

Re: The Magic of strace

#80

Post by brodney »

Xdes wrote: post #65
brodney wrote: post #64

Earlier quoted context omitted.

Read this: http://blog.bignerdranch.com/1907-hooked-on-dtrace-part-1/

Or you could just give a concise definition: >What is this "DTrace" thing? It stands for "Dynamic Tracing", >a way you can attach "probes" to a running system >and peek inside as to what it is doing.

It was my mistake to link you to a tutorial on the thing you are asking about. I don't know what came over me.
Post reply on HN