Live data from Hacker News

An easier way to use Python's cProfile

github.com

1–9 of 9 posts

Re: An easier way to use Python's cProfile

#4
I found that writing cProfile's profiling data to a file and analysing that with either RunSnakeRun's[1] or KCacheGrind's[2] GUI gave me much more insight than just the tabular display from cProfile. It gives you a really good visual overview of where your code spends its time and has a really good interface for drilling down into subroutines to pinpoint the exact location of the slow code.

[1] http://www.vrplumber.com/programming/runsnakerun/ [2] http://kcachegrind.sourceforge.net/html/Home.html

Re: An easier way to use Python's cProfile

#5
post #4

I found that writing cProfile's profiling data to a file and analysing that with either RunSnakeRun's[1] or KCacheGrind's[2] GUI gave me much more insight than just the tabular display from cProfile. It gives you a really good visual overview of where your code spends its time and has a really good interface for drilling down into subroutines to pinpoint the exact location of the slow code. [1] http://www.vrplumber.c…

Still not exactly great. I'd like to get a flamegraph view of the profile[0] or a chrome-style time series as it provides a much clearer high-level picture, but I'm not sure cProfile records enough information to do so (or that it stores threading informations in the profile, which would be necessary for time series)

[0] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

Re: An easier way to use Python's cProfile

#6
post #5
post #4

I found that writing cProfile's profiling data to a file and analysing that with either RunSnakeRun's[1] or KCacheGrind's[2] GUI gave me much more insight than just the tabular display from cProfile. It gives you a really good visual overview of where your code spends its time and has a really good interface for drilling down into subroutines to pinpoint the exact location of the slow code. [1] http://www.vrplumber.c…

Still not exactly great. I'd like to get a flamegraph view of the profile[0] or a chrome-style time series as it provides a much clearer high-level picture, but I'm not sure cProfile records enough information to do so (or that it stores threading informations in the profile, which would be necessary for time series) [0] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

I added flamegraph to plop, if you're interested:

https://github.com/andreasf/plop/tree/flamegraph_support

Here's an example: http://zentrale1.com/~an/nachher5.svg

Re: An easier way to use Python's cProfile

#8
post #6
post #5

Earlier quoted context omitted.

Still not exactly great. I'd like to get a flamegraph view of the profile[0] or a chrome-style time series as it provides a much clearer high-level picture, but I'm not sure cProfile records enough information to do so (or that it stores threading informations in the profile, which would be necessary for time series) [0] http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html

I added flamegraph to plop, if you're interested: https://github.com/andreasf/plop/tree/flamegraph_support Here's an example: http://zentrale1.com/~an/nachher5.svg

This thread changed my life.

Re: An easier way to use Python's cProfile

#9
post #8
post #6

Earlier quoted context omitted.

I added flamegraph to plop, if you're interested: https://github.com/andreasf/plop/tree/flamegraph_support Here's an example: http://zentrale1.com/~an/nachher5.svg

This thread changed my life.

Few more tips for successful Python profiling:

* By default, the ITIMER signals used by the profiler interrupt syscalls. Disable that by adding the following in plop.Collector.__init__() after the call to signal.signal():

  signal.siginterrupt(sig, False)
* Try all ITIMER modes, e.g. by changing the default in plop.Collector.__init__()