Earlier quoted context omitted.
Context: We use Jupyter heavily (mostly against Spark). In my experience there is a set of things that "traditional" Jupyter notebooks does really well. Anytime you have a linear flow of steps the notebook metaphor works really well. However, if you are doing things approaching traditional development, where you have multiple sources of data, or loops that require debugging, or basically anything that isn't linear in…
My experience is in line with yours, debugging loops and functions is a big pain point. However, I think there's a much better solution to be had here, which is to add more powerful debugging capabilities to Notebook. I think Notebook has potential for new debugging paradigms, imagine for example being able to break anywhere in a cell and get a new 'forked cell' which operates in the context of the code that you just…
To me, the traditional IDEs do work well for debugging and software development.
Notebooks are great for explanatory examples and interactive experiments. I think these are different to the type of software development I do when I use an IDE.
For example, I find notebooks great for rapid iteration of parameters when I'm doing "data science", or indeed most of the feature extraction->modelling->prediction data science pipeline.
What I don't find them good for is developing new algorithms. It isn't clear to me if this is an inherit limitation of the notebook format, or just something where it needs new developments.
(To be clear, I've also used both Zeppelin and Beaker notebooks and don't see any particular advantages. I've also used R Studio, but I don't really know enough R to comment sensibly on that)
