Live data from Hacker News

Debugging a Mixed Python and C Language Stack

developer.nvidia.com

1–10 of 25 posts

Re: Debugging a Mixed Python and C Language Stack

#3
post #2

GDB can decode Python stack traces as Python code, as long as you have have python-gdb.py from the Python source distribution in the same place as your Python executable.

Wow, that's pretty amazing. I wonder how it's implemented, and if there are any tutorials on implementing something similar, for programming language designers/creators.

Re: Debugging a Mixed Python and C Language Stack

#4
When developing PyTorch, we also run into a lot of mixed Python/C++ language situations. We've recently been experimenting with in-process 'combined' Python/C++/PyTorch 2.0 stack traces to make it easier to understand where code is executing (https://dev-discuss.pytorch.org/t/fast-combined-c-python-tor...).

Re: Debugging a Mixed Python and C Language Stack

#5
post #4

When developing PyTorch, we also run into a lot of mixed Python/C++ language situations. We've recently been experimenting with in-process 'combined' Python/C++/PyTorch 2.0 stack traces to make it easier to understand where code is executing ( https://dev-discuss.pytorch.org/t/fast-combined-c-python-tor... ).

Funny, I was going to post this, but the horse's mouth is even better ;)

Re: Debugging a Mixed Python and C Language Stack

#6
That reminds me of the time I debugged some code (also neural network) that was in both Java and C++ and I was able to attach both gdb and jdb to the same process but had to disable the segfault trap on gdb because the jvm segfaults all the time in normal operation.

Re: Debugging a Mixed Python and C Language Stack

#7
post #2

GDB can decode Python stack traces as Python code, as long as you have have python-gdb.py from the Python source distribution in the same place as your Python executable.

Wow, that's pretty amazing. I wonder how it's implemented, and if there are any tutorials on implementing something similar, for programming language designers/creators.

Theres also Py Spy, a profiling tool that can generate flame charts containing a mix of python and C (or C++) calls.

https://github.com/benfred/py-spy

It's worked really well for my needs

Re: Debugging a Mixed Python and C Language Stack

#9
An inspiring, albeit daunting, write-up for someone like me, who has been a python coder for over 10 years, but who has never professionally coded in C or C++. I'd be pleasantly surprised with myself if I could one day debug like that.

Also somewhat depressing that, yet again, the GIL was to blame, and that after all that impressive investigatory work, the fix was (spoiler alert!) "rewrite the offending function in C".

Post reply on HN