Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Guide to Python Debugging

martinheinz.dev

Post reply on HN
Print view View original

41–50 of 149 posts

Re: Guide to Python Debugging

#41

Post by albertzeyer »

To extend on inspecting stack traces: I found that I often wanted to know the the content of (relevant) (local) variables of a stack trace. Whenever I got a stacktrace, I would often restart it in a debugger, or add print statements on variables. Until I extended the stacktrace to just include this information. The tricky part is that printing all local variables is too much. And also global variables might be involved. Or attributes (e.g. self.x or so). So now I'm parsing all variables and attributes from the source code line from the traceback (in a very simple way) and print only those. This covers about 95% of all cases - i.e. in 95% of the case, the stack trace contains all information I need to understand and fix the problem.

I published that here:

https://pypi.org/project/better_exchook/ https://github.com/albertz/py_better_exchook

I also often have a SIGUSR1 handler which will print the stacktrace of all threads. This is useful on long running processes, involving multi threading, where you might run into some strange hangs or deadlocks.

In addition to that, if you might get crashes (segfault or so), something like faulthandler is useful.

If you also want to see the C stack trace in addition in such cases, I load libSegFault.so, like here: https://github.com/rwth-i6/returnn/blob/5b8e34ec1fd725d0e20b...

Profiling is another topic. Something like Py-Spy (https://github.com/benfred/py-spy) can be very helpful.

I also found a remote background ZMQ IPython/Jupyter kernel to be useful sometimes. I published that here: https://github.com/albertz/background-zmq-ipython

Re: Guide to Python Debugging

#42

Post by jedberg »

_coveredInBees wrote: post #3

This is a bit of a weird article. It spends most of the time talking about logging, which is somewhat useful for debugging but not really. pdb gets a few lines of description, and that is about it. Personally, I can't live without Pycharm when working with Python purely because of how fantastic the debugging experience is. The integration with the interactive IPython shell is simply fantastic, and the live variable v…

> Between being able to prototype code in the interactive IPython REPL and then graduate it seamlessly to scripts/functions/classes, and then having an amazing debugging UX to fallback on for those bugs that slip through,

Do you have a good tutorial on how this all ties together? It sounds like heaven but it's not how I use PyCharm because I don't know any better.

Re: Guide to Python Debugging

#43

Post by dijit »

markus_zhang wrote: post #15
_coveredInBees wrote: post #10

Earlier quoted context omitted.

Not the person you replied to, but I love vscode from when I did a bunch of React + JS stuff a few years back. I also do a ton of software development and machine learning work in Python now and have never managed to move away from Pycharm even though I'd love to be able to have a good enough experience in Vscode so that I could make the switch from Pycharm (the remote development experience in vscode is really neat…

Agreed to every point. Pycharm really shocked me by correctly suggesting most of library related stuffs as ling as I type hint them. I really enjoyed pycharm and datagrip and wondering how does clion compare with vs.

I've been using CLion with the 'rust' plugin, which is quite nice.

But it's not pycharm; the debugger is very much lacking, often failing to resolve or step through.

But this is likely because Rust is not a primary target for the IDE.. I haven't tried any complex C++ in it yet.

Re: Guide to Python Debugging

#45

Post by underdeserver »

vonseel wrote: post #32
_coveredInBees wrote: post #3

This is a bit of a weird article. It spends most of the time talking about logging, which is somewhat useful for debugging but not really. pdb gets a few lines of description, and that is about it. Personally, I can't live without Pycharm when working with Python purely because of how fantastic the debugging experience is. The integration with the interactive IPython shell is simply fantastic, and the live variable v…

Yeah I agree entirely on Pychaem debugger being awesome. But it’s no substitute for logging in production. Somethings just need to be logged, either for proof or those situations where you just can’t reasonably attach a Pycharm debugger.

Yes, logging is crucial for knowing what happened, but if you don't understand how it could have happened, step by step debugging is immensely useful. Assuming, of course, you can reproduce in your dev environment.

Re: Guide to Python Debugging

#46

Post by zo1 »

zmmmmm wrote: post #23

I feel like I got in a time machine ... this like debugging from 1995. In fact, pretty sure I would have rejected this in 1995. Unless you are constrained to doing this on a server or other restricted environment where reproducing locally is not possible, just get a good IDE and set some breakpoints in there.

The environment is very "VSCode" focused at the moment, hence the roundabout and inefficient ways to do actual debugging. Debugging in a proper python IDE is a breeze and on-par if not better to other traditional IDEs for staticly-typed languages.

Re: Guide to Python Debugging

#48

Post by mjbrownie »

I recommend repr in all log messages too. For a work Django project we have common BaseModel repr(obj) expression that can be easily converted (using regexes in loki and promtail) into a clickable admin link.

So within loki (or any similar log tools) all logger.info(repr(object)) is a link straight to the admin. Worth the set up time.

Re: Guide to Python Debugging

#50

Post by diarrhea »

_coveredInBees wrote: post #10
thomk wrote: post #6

Earlier quoted context omitted.

I'm a vscode user and I do like it. I only really use it for the file navigation and color coding though. Autocomplete drives me nuts, so I turned that off completely. I am a pdb fan so i drop in to that when needed. I never really considered PyCharm to be honest, it feels like overkill for me. However your allegiance sounds interesting. Can you give some reasons why you like it so much?

Not the person you replied to, but I love vscode from when I did a bunch of React + JS stuff a few years back. I also do a ton of software development and machine learning work in Python now and have never managed to move away from Pycharm even though I'd love to be able to have a good enough experience in Vscode so that I could make the switch from Pycharm (the remote development experience in vscode is really neat…

I have used both softwares but stick with VSCode for Python, simply because it is very powerful to have one great editor for all projects, as opposed to one specialized IDE for each.

PyCharm is (AFAIK) monolithic in what it brings to the table, whereas VSCode is more modular. As such, some of your points are true, but not an inherit fault of VSCode, but probably of some settings/underlying tools used. Maybe these can get you a better experience:

3. Agreed with VSCode being lackluster, but switching to MS Language Server [0] has helped this a great deal (while having other downsides, like being slower).

5. VSCode also has a variable viewer/browser and an interactive shell (which cannot be IPython as of today, however [1])

6. Disagree: I forgot what VSCode uses per default, but I switched to `black` for formatting, which works very well (` "python.formatting.provider": "black"` in settings.json). For linting, I switched to `mypy` (which also handles type hints, which is helpful), which can be set via `Python: Select Linter` (CTRL+SHIFT+P). `mypy` seems to be powerful. It definitely catches method 1st arguments not being `self`.

0: https://github.com/microsoft/python-language-server

1: https://stackoverflow.com/questions/47701629/vs-code-run-ipy...

Post reply on HN