Live data from Hacker News

Memray – A Memory Profiler for Python

github.com

31–33 of 33 posts

Re: Memray – A Memory Profiler for Python

#31

Earlier quoted context omitted.

I'm probably not the average python programmer. But I normally just create two terminals (I have a tiling window manager) and in one I open a python file under /tmp/ write my code and execute it in the other terminal. I would probably use a REPL if it was integrated in my favorite editor ( https://helix-editor.com ). But everything else I tried was to "clunky" for me. Though I work with data scientists, and they love…

A its spooky reading someone with basically the exact same workflow as me! I use helix in the terminal, regularly opening up a split pane in tmux to either breakpoint in, or test out bits of code interactively. I'm not quite as organized as having two regular panes- I'll close and open them pretty quickly. Often just to try some toy example of reorganising a duct or something before writing it out into code.

Haha great to hear I'm not the only one! I just miss the speed of Helix when using Jupyterlab et al, so I just do it this way

Yeah I'm definitely not that organized, I also don't keep both open all the time, my fingers are sometimes to quick and close one of the terminals without me thinking about it. But I kept it simple in my example, so others could get the idea, as this is basically the "concept" behind my workflow.

Re: Memray – A Memory Profiler for Python

#32
post #10

For those Python programmers out there, if you don't mind sharing your experiences, do you spend any time at all on the REPL? If so, what fraction of the time, approximately? Using IPython, JupyterLab, or something else? Or do you just run it directly from an VS Code or PyCharm? Anything you may want to add about your routine would be appreciated. Oh, if you (experienced programmer or not) happen to know about a good…

I use “breakpoint()” a lot for debugging. It’s by far my #1 tool for figuring out why something isn’t working. Recommend you learn how to use it.

Thank you so much. Your comment helped me a lot. I wish a lot more people knew about this.

Re: Memray – A Memory Profiler for Python

#33

I collected a list of profilers (also memory profilers, also specifically for Python) here: https://github.com/albertz/wiki/blob/master/profiling.md Currently I actually need a Python memory profiler, because I want to figure out whether there is some memory leak in my application (PyTorch based training script), and where exactly (in this case, it's not a problem of GPU memory, but CPU memory). I tried Scalene ( htt…

It is still in experimental stages, but would you be willing to test the memory mode of Echion? (https://github.com/P403n1x87/echion). It is essentially tracing of all memory allocations. Mind that, if an allocation is followed by a matching de-allocation, it won't be reported; so if you profile from start till end, and all the memory is released, the profile should be almost empty (in reality this will almost never be the case); this might explain why the data doesn't seem to make sense from the tools you have tried already (assuming they do a similar thing).
Post reply on HN