21–30 of 74 posts
This looks nice and all, but on the other hand it's kind of depressing how people are expending all this effort on Jupyter and surrounding stuff, just to make something that almost, but not quite as good as Smalltalk.
I always thought a debugger was more useful for when you got a big buried in a long execution sequence and produced by unknown combinations of variables. Where as usually my notebook cells are only a few lines long, and with a very clear state going in. Can any of you speak to the practical need for a debugger in Jupter?
That is only true in one very specific case: When you are executing all code cells in order, from top to bottom, exactly once.
If you start executing cells multiple times, or out of order, or even worse, execute only parts of cells (which is possibke in many Jupyter UIs), all bets are off. Anything can happen.
I'll take this opportunity to get jupyter advice: one thing I don't understand about jupyter vs jupyterlab is why notebooks in jupyter classic have functioning vim bindings (ie in cells) through the codemirror extension but jupyterlab notebooks do not. in jupyterlab you can have vim bindings in the text editing view but not in the notebook view. for the life of me I cannot understand this design decision. how hard ca…
This looks nice and all, but on the other hand it's kind of depressing how people are expending all this effort on Jupyter and surrounding stuff, just to make something that almost, but not quite as good as Smalltalk.
I'll take this opportunity to get jupyter advice: one thing I don't understand about jupyter vs jupyterlab is why notebooks in jupyter classic have functioning vim bindings (ie in cells) through the codemirror extension but jupyterlab notebooks do not. in jupyterlab you can have vim bindings in the text editing view but not in the notebook view. for the life of me I cannot understand this design decision. how hard ca…
there is an extension to provide vim bindings within cells- jupyterlab-vim
Everyone should aim to minimize the amount of work they do in Jupyter Lab / Notebook.
It shocks me a bit to find myself saying that, as it is such a beautiful piece of work. Furthermore the people who wrote it are better software engineers than I'll ever be: the frontend, the zeromq-mediated communication with the kernel, the fact that the architecture has generalized so successfully to other language kernels, its huge popularity and reach. Nevertheless, I believe I'm serious. It really comes down to just two related issues, but they're extremely important: debugging and version control.
If you're a software engineer, and not a data scientist, here's how you probably debug already, or if not then how you should debug:
- You identify (a) commit(s) on which the behavior is correct, and (a) commit(s) where it is not correct.
- You experiment with fixes. Perhaps you stash them, perhaps you create experimental commits.
The critical point is that you use your version control system (probably Git) to navigate between alternative versions of the code. With a single command, you can switch the version of your code base, and the subsequent process you invoke to test your code is a fresh process, unpolluted by any state from the version of your code that you were on 30 seconds ago.
In contrast, Jupyter notebook does not encourage this style of work at all. In practice, what you will do when trying to debug some code in Jupyter is comment out lines, temporarily delete code, add experimental lines, add experimental new cells, etc. All creating a working tree, and a collection of in-memory python objects, that is a baffling mixture of changes related to the original feature development, and changes related to experimental debugging. Debugging will wear you out, as the state of your notebook gradually approaches complete incomprehensibility.
If you're a software engineer, you'll already know the benefits of being able to make precise adjustments to the state of your code with git commands. You want to learn statistics and data analysis skills from data scientists, but in doing so you should not regress to a worse style of development by starting to write much of your code in Jupyter notebooks.
And if you're a data scientist, you will want to acquire the debugging skills of software engineers. If you are not using git, you want to start learning it now.
Crudely, we can imagine a 2-dimensional diagram with one axis for engineering skills and another for data science skills. Everyone wants to be in the top-right quadrant. In that quadrant, version control is used, and the version control system is used for debugging. Debugging is rather important in developing all software, whether scientific/numerical or not.
So both groups should be minimizing the amount of code written in the Jupyter notebook UI: instead, write code in a standard Python package, in a virtualenv, installed in editable mode with `pip install -e`. If you need to use a notebook for graphical display, or HTML display of Pandas dataframes, or display of an audio playing widget, or any of the other amazing things it does so well then fine: use importlib.reload in your notebook to load and reload the bulk of your code from your Python package. The notebook should just feature calls to plotting routines etc that you have implemented in standard code files using your text editor/IDE. You could even aim for your notebook to contain so few lines of code that in some projects you might not even bother committing it.
I'm a programmer, mostly not a data scientist nowadays, mostly working with Python. I have tried Jupyter Lab/Notebook on and off over the last 10 years, and I believe I have now firmly settled on my conclusion: Everyone should aim to minimize the amount of work they do in Jupyter Lab / Notebook. It shocks me a bit to find myself saying that, as it is such a beautiful piece of work. Furthermore the people who wrote it…
Many of our PhD students learned programming using matlab and it's a mess, they never use version control, everything is in a single script, nobody properly debugs, etc.. I believe this is because matlab encourages that type of programming.
I decided very early on to use python instead of matlab, significantly before ipython notebooks became a thing. Because most of the python resources came from computer scientists, using software engineering methods was really "forced" onto me, and I really enjoy it now.
If I look at the people who have been converted to python using jupyter I see a very similar phenomenon that you talk about. People are creating a huge mess, in ways it's even worse than the monolithic matlab scripts, because the notebooks can't even be run as a whole unit, because working and not working cells are mixed, there are several cells that define the same function, but only one is the correct one (good luck remembering which one it is after a couple of months) ...
I thing jupyter is a great tool for teaching and exchanging and presenting analyses. But it is a terrible tool for programming and especially learning to program, because it really encourages bad practices.
I don't see how this debugger is visual, but anyway it's cool to see that jupyter finally has a debugger. This is what I would call a visual debugging experience: https://github.com/hediet/vscode-debug-visualizer/blob/maste... (Disclaimer: I'm the author of that extension)
I don't see how this debugger is visual, but anyway it's cool to see that jupyter finally has a debugger. This is what I would call a visual debugging experience: https://github.com/hediet/vscode-debug-visualizer/blob/maste... (Disclaimer: I'm the author of that extension)
cool extension but literally every other use of the term visual debugger refers to a graphical interface for something like gdb (where you have to manually type in commands to set break points, step, continue, etc). so I don't think it makes sense to claim that this is not a visual debugger simply because you've decided to redefine the term to suit your needs (publicizing your extension).
Thus, people in one camp see this as the standard name for the feature, and people in the other camp view it as misleading, and a continued diluting of the word in a way which trivializes their research.
After a decade of hearing "oh, visual programming, you mean like Visual C++?", I've learned to avoid the word entirely. It's a loaded term.
Every common word that is used as a popular brand name has this problem. I'm getting flashbacks to the 80's/90's and trying to explain my home computer to IBM PC people. "Do you have Windows?" "Well, it's an Apple. There are windows on the screen but it's not Microsoft Windows." "If you've got windows on the screen you can drag around with a mouse, that's Microsoft Windows."
Who cares? Once you're editing text in a browser you've failed.