Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

github.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#61

Post by mooneater »

jchw wrote: post #48
mooneater wrote: post #47

Earlier quoted context omitted.

Wow your suggestion really helped: Debug toolbar itself was causing this for some reason. How embarrassing. Thanks!

That is rather surprising, because I don't have the same issue with a pretty large production app. I wonder if there's a deeper underlying issue.

Ok I upgraded toolbar from 1.7 to 1.8 and problem is gone :)

There are a number of toolbar bugs at https://github.com/jazzband/django-debug-toolbar/issues/943 involving keywords "slow" and "hang", not sure which it was.

But the meta-issue is, the profiling tools I used were not super helpful in discovering what the issue was.

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#62

Post by willangley »

alex- wrote: post #25
galonk wrote: post #11

I got excited but then read that it's some kind of cloud-based web application thing. Is there something like this (show memory use and call times for a Python process) that just runs on my computer to help me profile a long-running Python process?

psutil ( https://pythonhosted.org/psutil/ ) is awesome for collecting valuable monitoring information. pyrasite ( http://pyrasite.com/ ) will let you inject code into a process. This can be used to add monitoring of private internal state etc (if you have no other options). If you want to have locally hosted graphs then grafana and influx are my current tools of choice. It is going to be more work than swiping a cred…

pyrasite can make the child process slow down markedly; this came up for me when injecting profilers into cinnamon-screensaver to try to root cause a memory leak (https://bugs.launchpad.net/linuxmint/+bug/1652489)

cinnamon-screensaver would take multiple seconds to lock the screen even after I'd stopped profiling and exited the interpreter I'd injected, and I wound up restarting it so I could lock my screen quickly again.

I don't know why this happened, but it's enough to make me think twice, and I'm definitely going to double-check my process is still performing as I hope after injecting it with pyrasite in the future.

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#63

Post by opticalfiber »

dmitrim wrote: post #51
opticalfiber wrote: post #49

Can you comment on how compatible this is with asyncio-based applications? Looks like an interesting product and I'm going to try it out regardless, but it would be cool to get some clarity since I couldn't find anything in the docs besides this: > Time (blocking call) profiler supports threads and gevent. (from https://stackimpact.com/docs/#getting-started-with-python-pr... )

We haven't tested the whole agent with asyncio applications yet. I guess only CPU profiler was tested during development. We'll do and include it in the docs. For now, if you see any problems, please just open a ticket. Thanks!

First! https://github.com/stackimpact/stackimpact-python/issues/1 :D

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#64

Post by icebraining »

cookiecaper wrote: post #55
metalliqaz wrote: post #3

This is an interesting project. It appears to me, based on the README and the name, that it was primarily intended to profile backend services in a web app. I have a question for hacker news. Does Python still have a lot of momentum in this area? I love Python and use it whenever I can, but I find these days that most web frameworks assume right off the bat that you are using Node. The frontend landscape is so heavil…

IMO having some Node.js experience and React experience are very helpful to get hired into any position that is remotely near the web these days. I have seen several of my clients introduce Node.js and of course everyone is introducing SPAs in some form or another, mostly React. Mozilla is even integrating React into Firefox's UI. I am personally skeptical of these trends, but my skepticism doesn't change the shift o…

Which new languages are you referring to?

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#65

Post by pietjepuk88 »

galonk wrote: post #11

I got excited but then read that it's some kind of cloud-based web application thing. Is there something like this (show memory use and call times for a Python process) that just runs on my computer to help me profile a long-running Python process?

I also needed to profile a running process without stopping it, and ended up using pyflame (https://github.com/uber/pyflame) together with the famous flamegraph script. It is not perfect, as it only works on Linux and only does CPU profiling, but it worked well enough for my purposes.

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#66

Post by fnl »

galonk wrote: post #11

I got excited but then read that it's some kind of cloud-based web application thing. Is there something like this (show memory use and call times for a Python process) that just runs on my computer to help me profile a long-running Python process?

As nobody yet has mentioned it: lptrace -> strace for Python programs @ https://github.com/khamidou/lptrace

For Python3: https://github.com/rixx/lptrace/tree/python3

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#67

Post by metalliqaz »

icebraining wrote: post #64
cookiecaper wrote: post #55

Earlier quoted context omitted.

IMO having some Node.js experience and React experience are very helpful to get hired into any position that is remotely near the web these days. I have seen several of my clients introduce Node.js and of course everyone is introducing SPAs in some form or another, mostly React. Mozilla is even integrating React into Firefox's UI. I am personally skeptical of these trends, but my skepticism doesn't change the shift o…

Which new languages are you referring to?

I'm willing to bet that one of the ones he was thinking of is Go

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#68

Post by metalliqaz »

cookiecaper wrote: post #55
metalliqaz wrote: post #3

This is an interesting project. It appears to me, based on the README and the name, that it was primarily intended to profile backend services in a web app. I have a question for hacker news. Does Python still have a lot of momentum in this area? I love Python and use it whenever I can, but I find these days that most web frameworks assume right off the bat that you are using Node. The frontend landscape is so heavil…

IMO having some Node.js experience and React experience are very helpful to get hired into any position that is remotely near the web these days. I have seen several of my clients introduce Node.js and of course everyone is introducing SPAs in some form or another, mostly React. Mozilla is even integrating React into Firefox's UI. I am personally skeptical of these trends, but my skepticism doesn't change the shift o…

Thanks, good stuff to think about.

Do you think React is something to learn together with Node or would you recommend just getting to know React on its own?

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#69

Post by cookiecaper »

metalliqaz wrote: post #68
cookiecaper wrote: post #55

Earlier quoted context omitted.

IMO having some Node.js experience and React experience are very helpful to get hired into any position that is remotely near the web these days. I have seen several of my clients introduce Node.js and of course everyone is introducing SPAs in some form or another, mostly React. Mozilla is even integrating React into Firefox's UI. I am personally skeptical of these trends, but my skepticism doesn't change the shift o…

Thanks, good stuff to think about. Do you think React is something to learn together with Node or would you recommend just getting to know React on its own?

Bolting React onto an existing codebase is probably best because it is more similar to what you'd see in the real world. Most people don't start a React/Node thing at the same time; they'll start integrating React into their frontend early because they can get little React-compatible widgets plugged in more easily than they can introduce backend changes like finding opportunities for Node.

React and Node are not really related other than they're both JavaScript-based, so the skills aren't really co-dependent. Node is used to execute JavaScript locally (in build tools like Webpack, for example), but beyond a small amount of local scripting for builds, they don't really touch (afaik; I have not yet completed a major project with either of them, just used them here and there).

Re: Show HN: StackImpact – Python Production Profiler: CPU, Memory, Exceptions

#70

Post by activatedgeek »

sciurus wrote: post #41
activatedgeek wrote: post #23

Now that this has come up, can somebody explain me how do profilers work? My main concern being regarding the overhead to the process it adds.

There are two approaches to profiling: Deterministic profiling monitors every function call to track timing information. This is very precise but adds significant overhead. Statistical profiling samples the call stack periodically to see what functions are running. This is less precise but has less overhead. The overhead varies depending on how frequently you sample and what the sampling mechanism is. StackImpact is…

Fair enough. Thanks.
Post reply on HN