why the dash, py-spy vs rbspy?
Show HN: Py-spy – A new sampling profiler for Python programs
31–40 of 44 posts
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#32Wonderful. Can the data it produces be munged into something KCachegrind can show?
Not yet - but I'm hoping to have a version that supports this next week. Will update this issue when it's done: https://github.com/benfred/py-spy/issues/3
Great, thanks!
And as a slightly different take than that of the person posting the issue - interfaces like kcachegrind are a pretty clunky (if powerful, in their clunky way) - the profiler coming with some built-in presentation and reporting of its own like the flamegraph and the realtime display is a big win and a serious deficiency in most python profilers.
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#33Does anyone know if there is something like this for Nodejs? Of course you can enable profiling, but it would be nice to be able to look at running processes.
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#34I've been enjoying pyflame from Uber - which the author quotes in their info
> The only other Python profiler that runs totally in a separate process is pyflame, which profiles remote python processes by using the ptrace system call. While pyflame is a great project, it doesn't support Python 3.7 yet and doesn't work on OSX or Windows.') > Py-spy works by directly reading the memory of the python program using the process_vm_readv system call on Linux, the vm_read call on OSX or the ReadProcessMemory call on Windows.
I think ptrace is fundamentally letting you do the same thing in terms of how pyflame is using it.. and the same ptrace access permission governs whether you can use process_vm_readv
The real win for this project is a real-time "top" or "perf top" style UI instead of only generating flamegraph output. I love that feature, and will be particularly good for quick shot "what is this process doing" type info as opposed to specifically profiling some timeframe to analyse the resulting flamegraph (which is all pyflame let you do)
Nice work!
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#35This is a really great tool - the kind I didn't even know that I needed until I was given it!
One question, does anyone here know how to interpret the GIL (Global Interpreter Lock) percentage display in the top-left? In my code, "Active" sticks nicely at 100%, but the GIL jumps around from 1% to 100%, changing on every sample.
edit: Now that I think about it, my code spends a lot of time in C API calls - maybe the GIL is released there?
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#36Many thanks for building and releasing this. It's ridiculously easy to install (especially in virtualenvs) and very powerful. When I push '3' or '4', I get informative, stable output. Minor feature request: an explicit 'pause' button would make it easier to copy file paths from the output. Ctrl-S is a reasonable alternative, but it's a little hacky. Also, it would be nice to somehow eliminate time spent in poll() fro…
Thanks! Both of your suggestions totally make sense. I've created an issue to track the poll() issue here https://github.com/benfred/py-spy/issues/13 - I think that should be an easy fix.
I think a more robust solution might be to have counters for in-Python samples, outside-Python, and in-syscall.
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#37This is a really great tool - the kind I didn't even know that I needed until I was given it! One question, does anyone here know how to interpret the GIL (Global Interpreter Lock) percentage display in the top-left? In my code, "Active" sticks nicely at 100%, but the GIL jumps around from 1% to 100%, changing on every sample. edit: Now that I think about it, my code spends a lot of time in C API calls - maybe the GI…
Wow, it tracks GIL contention? Major feature for me. I use a lot of Numba, and it releases the GIL if you want — so threading is actually useful.
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#38why the dash, py-spy vs rbspy?
Just a guess but remove the dash and read it out loud.
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#39Earlier quoted context omitted.
I've never understood why flame graphs are better than the normal presentation of inclusive and exclusive timings in performance tools, even if they're not "modern", but embody some decades' experience. Anyone care to explain?
Flame graphs have extra information in the form of seeing the caller and callees. That, and I find the graphical presentation to be easier to scan.
Of course I expect to see the call tree, and to flip between that and a flat view, to flip between different metrics in the views, and to see them across processes/threads. I'm talking about graphical tools (perhaps coupled with external reduction of the data) like CUBE [1], Paraprof [2], and those in toolsets like Open|Speedshop [3] and HPCToolkit [4] with which I'm less familiar.
1. http://www.scalasca.org/software/cube-4.x/documentation.html
2. https://www.cs.uoregon.edu/research/tau/docs/newguide/bk01pt...
Re: Show HN: Py-spy – A new sampling profiler for Python programs
#40Earlier quoted context omitted.
I've never understood why flame graphs are better than the normal presentation of inclusive and exclusive timings in performance tools, even if they're not "modern", but embody some decades' experience. Anyone care to explain?
I'm far from a performance expert, but my impression is: It shows the call paths to the functions and what part each path took, that's not so obvious from the typical table. On the other hand, finding functions that are called quite a lot all over the place and add up is easier in the table, so it's not become useless.
Indeed, you want alternative views depending on what sort of profile or question about it you have -- with toggles in the GUI.