PyInstrument – A statistical Python profile that focuses on the slow parts
1–10 of 23 posts
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#2>
The standard Python profilers profile and cProfile show you a big list of functions, ordered by the time spent in each function. This is great, but it can be difficult to interpret why those functions are getting called. It's more helpful to know why those functions are called, and which parts of user code were involved.
Note that you can use something like gprof2dot to convert pstats dump from cProfile to a visual callgraph: https://github.com/jrfonseca/gprof2dot#python-cprofile-forme...
Not saying that solution’s better than pyinstrument — I haven’t use this one before so I’ll have to evaluate. Also, the lower overhead is undeniable.
---
Edit: Another thing I noticed in "How is it different to profile or cProfile?":
> 'Wall-clock' time (not CPU time)
> Pyinstrument records duration using 'wall-clock' time. ...
Seems misleading as cProfile uses time.perf_counter unless you supply your own timer, and time.perf_counter does measure wall clock time. See
https://github.com/python/cpython/blob/ec42789e6e14f6b6ac135...
https://docs.python.org/3/library/time.html#time.perf_counte...
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#3Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier.
As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is super useful when your program is stuck and you don't know why. I haven't used pyinstrument yet but I do like the fact that it can do its flame graph in the console, sometimes I find saving down an svg file and opening up the browser a bit arduous. Excited to give it a try.
[1] https://github.com/benfred/py-spy
[2] https://jvns.ca/blog/2018/09/08/an-awesome-new-python-profil...
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#4Does this also track time spent in cython modules and function calls through ctypes?
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#5> Shows you why your code is slow!
Because you wrote it in Python.
Seriously, Python is probably the slowest mainstream language of all. If you’re building something where performance matters, you should be using a different language.
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#6> Shows you why your code is slow! Because you wrote it in Python. Seriously, Python is probably the slowest mainstream language of all. If you’re building something where performance matters, you should be using a different language.
_Ruby enters the room_
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#7Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier. As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is…
py-spy seems also very interesting, thanks for linking it. I always found cPython to be quite difficult to work with, and usually reverted to line_profiler or some sort of UI for cprof files.
The main added benefits of pyInstrument to me is the high signal vs noise ratio, as it is evidently clear what is taking the most time while retaining the option to dive deeper.
I am also curious to try out the on-demand profiling integration with Flask, seems like a cool thing to have running in the background for my side projects
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#8> Shows you why your code is slow! Because you wrote it in Python. Seriously, Python is probably the slowest mainstream language of all. If you’re building something where performance matters, you should be using a different language.
So you don't make bad performance choices in C?
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#9> Shows you why your code is slow! Because you wrote it in Python. Seriously, Python is probably the slowest mainstream language of all. If you’re building something where performance matters, you should be using a different language.
> Because you wrote it in Python.
Sure, so if I want to shave startup/a slow action from 200ms to 100ms in a non-performance-critical tool, I shouldn't use a profiler, I should rewrite the whole damn thing in Go?
Can we stop these low information, canned responses already.
Re: PyInstrument – A statistical Python profile that focuses on the slow parts
#10Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier. As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is…
Another relatively new addition to the python statistical sampler space is Austin[1], that has a lot of similar features to py-spy. I haven't made a direct comparison yet between the two.