Memray: a memory profiler for Python
github.com
Memray: a memory profiler for Python
1–10 of 53 posts
Re: Memray: a memory profiler for Python
#2[deleted]
Re: Memray: a memory profiler for Python
#3Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases?
My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exactly easy to identify memory usage of codepaths.
Re: Memray: a memory profiler for Python
#4Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exa…
I've had good luck with yappi for debugging asynchronous code, not really focused on memory usage but walltime & such.
Re: Memray: a memory profiler for Python
#5A twitter thread with some screenshots and details - https://twitter.com/1st1/status/1516859294896906241
Re: Memray: a memory profiler for Python
#6Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exa…
Do you mean specifically for Python? There's this for async Rust:
Re: Memray: a memory profiler for Python
#7Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exa…
I've had good luck with yappi for debugging asynchronous code, not really focused on memory usage but walltime & such. https://pypi.org/project/yappi/
Thanks, that looks useful!
Re: Memray: a memory profiler for Python
#8Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exa…
I like open source Perfetto UI (formerly Google Chrome about::/tracing)
Wrote my own timing trace json export for Python and C++, across threads and processes. The docs have some pointers to creating your own traces with the Tracing SDK.
Re: Memray: a memory profiler for Python
#9Tangential question: Does anybody have a good recommendation for a profiler that works well with massively async codebases? My experience has been that the concurrent nature of coroutines can make it hard to reason about what's going on at a particular point in time. If you don't know how many things you're awaiting on at a specific moment (and what potential external stuff they may be interacting with), it's not exa…
I like open source Perfetto UI (formerly Google Chrome about::/tracing) Wrote my own timing trace json export for Python and C++, across threads and processes. The docs have some pointers to creating your own traces with the Tracing SDK. See https://ui.perfetto.dev
viztracer can create traces for python programs. Not sure about async-io awareness though.
Re: Memray: a memory profiler for Python
#10Earlier quoted context omitted.
I like open source Perfetto UI (formerly Google Chrome about::/tracing) Wrote my own timing trace json export for Python and C++, across threads and processes. The docs have some pointers to creating your own traces with the Tracing SDK. See https://ui.perfetto.dev
viztracer can create traces for python programs. Not sure about async-io awareness though.
Actually it has explicit support for async task based reporting: