FunctionTrace: Graphical Python Profiler • Hacker News Board

Re: FunctionTrace: Graphical Python Profiler

#31

Post by itamarst » Thu, Jul 06, 2023, 12:00 PM UTC

It's nice to see how many different approaches to profiling there are these days in Python. I work on another (commercial but with free plan) Python profiler, Sciagraph: https://sciagraph.com.

The main use case is data science and other long-running batch jobs. Some differences:

1. It does memory profiling at basically no performance overhead; sounds like for FunctionTrace it's high overhead so off by default. And it catches _all_ memory allocations, not just Python API ones. This is based on using sampling, so it's not useful for profiling tiny functions (but for data science/scientific computing it'll work just fine).

2. Uses sampling for performance profiling, unlike FunctionTrace. Again, perfectly fine for any non-micro-benchmark data science program.

3. Also has a timeline view, without having to upload your data anywhere.

4. No native stacks yet.

5. Shows you if you're using CPU or I/O for every particular sample.

Re: FunctionTrace: Graphical Python Profiler

#32

Post by Veserv » Thu, Jul 06, 2023, 3:31 PM UTC

itamarst wrote: post #31

It's nice to see how many different approaches to profiling there are these days in Python. I work on another (commercial but with free plan) Python profiler, Sciagraph: https://sciagraph.com . The main use case is data science and other long-running batch jobs. Some differences: 1. It does memory profiling at basically no performance overhead; sounds like for FunctionTrace it's high overhead so off by default. And i…

You should really not randomly speculate when doing adversarial comparisons. Note: I am not a author of FunctionTrace.

1. You claim ~5% overhead [1] and it is not useful for profiling tiny functions. They claim ~10% overhead when handling tiny functions and, who I believe to be the author of the package below, said it averages low single digit percents (i.e 2. Sampling is so much worse than a full execution trace from a performance optimization and observability perspective it is ridiculous. Yes, a magnifying glass and a microscope are both okay for looking at ants, but only one is good for looking at cells. People prefer sampling because it was traditionally lower overhead, but for the same cost full tracing is so much better it is not even worth comparing, the result is patently obvious.

3. No “upload” is done. You are not sending data to them, everything is local. They are just opening the profiling data using Firefox to use it as a viewer like how you might open a PDF in Firefox.

[1] https://www.sciagraph.com/docs/understanding/fil/

Re: FunctionTrace: Graphical Python Profiler

#33

Post by formerly_proven » Thu, Jul 06, 2023, 7:33 PM UTC

actinium226 wrote: post #25

How does this compare to snakeviz?

snakeviz, tuna, pyprof2calltree etc. use data collected by cProfile which is probably the most flawed profiler anyone has been shipping for a long time, because cProfile only records the caller instead of the stack. Strangely enough the documentation for cProfile has an entire "Limitations" section which does not mention this.

Recording only callers instead of stacks means you literally cannot distinguish different control flows going through a common function at some point. All tools, like the mentioned ones, which pretend to reconstruct a flame graph or similar visualization don't actually work. A common cause for degenerate cases are function decorators used by multiple functions, especially if the decorator appears multiple times on the stack.

So for many practical programs the output of these is simply wrong and misleading.