How to spy on a Ruby program • Hacker News Board

Re: How to spy on a Ruby program

#11

Post by chrisseaton » Sun, Jun 12, 2016, 2:04 PM UTC

ch4s3 wrote: post #9

chrisseaton wrote: post #5

Earlier quoted context omitted.

I'm not sure what you want to do. Is your problem that you do want to see the native Java classes, or you want to hide them and focus on the Ruby code? Or is the key thing you want a Java flame graph?

Java classes and a flame graph would be nice sometimes. I really only know how to use pry and Virtual VM with JRuby.

It looks like some people are using flame graphs for the JVM.

http://techblog.netflix.com/2015/07/java-in-flames.html

I'm still puzzled that there seems to be no simple gprof-style graph profiler included with the JVM.

Re: How to spy on a Ruby program

#12

Post by ch4s3 » Sun, Jun 12, 2016, 2:08 PM UTC

ch4s3 wrote: post #9

chrisseaton wrote: post #5

Earlier quoted context omitted.

I'm not sure what you want to do. Is your problem that you do want to see the native Java classes, or you want to hide them and focus on the Ruby code? Or is the key thing you want a Java flame graph?

Java classes and a flame graph would be nice sometimes. I really only know how to use pry and Virtual VM with JRuby.

Good find, thanks Chris. Yeah, you'd think the JVM folks would work to add some tooling like that.

Re: How to spy on a Ruby program

#13

Post by jon-wood » Sun, Jun 12, 2016, 2:08 PM UTC

If you're planning ahead you can have your application load rbtrace which then allows connecting to a running process to see what it's doing, with options to limit to slow calls, IO, of specific method calls.

Re: How to spy on a Ruby program

#14

Post by lamontcg » Sun, Jun 12, 2016, 3:10 PM UTC

When I've needed to dump a stack trace, I've just included an interrupt handler which prints the call stack to a debug log.

That has been good enough for the problem of "wtf is this ruby process doing for _minutes_ at a time?" That doesn't get you flame graphs, but you can take a few snapshots and get an idea of what is happening.

For more involved perf debugging I've used ruby-prof.

Re: How to spy on a Ruby program

#15

Post by xjlin0 » Sun, Jun 12, 2016, 3:29 PM UTC

Nice write. The FlameGraph link in Julia's article is broken, maybe points to this one? https://github.com/brendangregg/FlameGraph

Re: How to spy on a Ruby program

#16

Post by jzwinck » Sun, Jun 12, 2016, 3:31 PM UTC

Of course there are equivalent programs for C. On GNU/Linux it's pstack aka gstack. Also gcore if you want an entire core dump you can analyze later with gdb.

http://linux.die.net/man/1/pstack

Re: How to spy on a Ruby program

#17

Post by jsnk » Sun, Jun 12, 2016, 3:53 PM UTC

"I was going to paste the strace output of what gdb is actually doing, but it is 20 megabytes of system calls."

I think this is why you shouldn't run this on production server itself. Each call is very resource intensive on the production server.

I believe the right way to analyze memory is to use "gcore", dump the memory, download it to the local machine's VM instance that's running the same OS as the production using scp. Also download the same ruby binary that production is running, and use gdb on the VM to analyze memory dump.

Re: How to spy on a Ruby program

#18

Post by aardvark179 » Sun, Jun 12, 2016, 4:42 PM UTC

chrisseaton wrote: post #11

ch4s3 wrote: post #9

Earlier quoted context omitted.

Java classes and a flame graph would be nice sometimes. I really only know how to use pry and Virtual VM with JRuby.

It looks like some people are using flame graphs for the JVM. http://techblog.netflix.com/2015/07/java-in-flames.html I'm still puzzled that there seems to be no simple gprof-style graph profiler included with the JVM.

I"m surprised as well.

Last time I needed this I knocked something up that could use the internals of Mission Control or Visual VM to turn their profile formats into flame graphs, but I doubt it would keep working across versions, and it really needed more work to be something anybody else could use sensibly.

It was however very easy to do (maybe half an hour's work) and produced very useful results. If you're using an invokeDynamic based language implementation however you will want to filter a lot of internal stack frames out of the graph, or you'll have trouble seeing past the LambdaForms to what's really going on.

Re: How to spy on a Ruby program

#19

Post by jeffdavis » Sun, Jun 12, 2016, 4:51 PM UTC

jsnk wrote: post #17

"I was going to paste the strace output of what gdb is actually doing, but it is 20 megabytes of system calls." I think this is why you shouldn't run this on production server itself. Each call is very resource intensive on the production server. I believe the right way to analyze memory is to use "gcore", dump the memory, download it to the local machine's VM instance that's running the same OS as the production usi…

That may be "right" given the current options, but that doesn't mean we can't have better options.

And that's exactly what this blog post is about.

Re: How to spy on a Ruby program

#20

Post by parennoob » Sun, Jun 12, 2016, 5:17 PM UTC

From the article:

“I'm constantly surprised by how many people don't know you can do this. It's amazing.”

I'm probably nitpicking, but sad to see this in the article. One of the things I love about Julia's writing otherwise is that it is free of this sort of 'I'm surprised that you don't know this simple thing.' expressions.