You should never use notebooks - period. My TL;DR: • Confusing for beginners • Encourage bad practices • Poor editor
Why not?
My TL;DR:
• Confusing for beginners
• Encourage bad practices
• Poor editor
The Future of Notebooks: Lessons from JupyterCon
31–40 of 158 posts
Re: The Future of Notebooks: Lessons from JupyterCon
#32Earlier quoted context omitted.
Inline documentation and charting (which for data tasks is a honking huge deal). It's literate programming with an actual use-case. Very into this idea.
None of that is what people use bash for, however.
You'd be surprised. It's certainly not _all_ of what people use bash for, but there are a lot of sketchy ETL (and sampling, and small-scale analytics) jobs out there held together with bash and awk and hopes and dreams, and replacing those with something like this is a substantial win.
Re: The Future of Notebooks: Lessons from JupyterCon
#33Most comments here express disbelief and disappointment in Jupyter from the software engineering point of view. What exactly is wrong with it? I use Jupyter daily and find no other Python environment more productive, be it scripts or IPython or IDEs. Granted, I work in scientific computing and use Python for data wrangling and stats. I find immense value in interactivity and iteration speed.
Jupyter is also really useful for figuring out how you want to do something before you copy it to your IDE. Every time I work with a new API or library I try it out in jupyter first.
What are the benefits of this vs just an integrated repl?
Re: The Future of Notebooks: Lessons from JupyterCon
#34I understand why they became popular, but as a software engineer considering how they work, I am just full of disappointment. we're going to spend the next ten years re-inventing every single software engineering best practice for jupyter's weirdo environment.
This is definitely one of my concerns too. Ad hoc code inside these notebooks is almost completely unmanageable from any reasonable software maintenance perspective, and refactoring code out of them is prohibitively difficult as well. I really want something to emerge that combines the best of both worlds of an IDE and notebook development, but there isn't anything close currently.
Some simpler notebook-like environments stay closer to source code in that they basically are source code with interleaved results (e.g. as comments). IMHO they hit a sweet spot between REPLs and those notebook environments inspired by mathematica, maple and similar more mathematically oriented software products.
Re: The Future of Notebooks: Lessons from JupyterCon
#35Most comments here express disbelief and disappointment in Jupyter from the software engineering point of view. What exactly is wrong with it? I use Jupyter daily and find no other Python environment more productive, be it scripts or IPython or IDEs. Granted, I work in scientific computing and use Python for data wrangling and stats. I find immense value in interactivity and iteration speed.
I don't think there's anything inherently wrong with jupyter notebooks, but they run in the browser without behaving in the same way a webpage or web app will in terms of managing state.
This is sort of tantamount to opening a website in your browser, but as soon as it loads, you need to refresh it as to "correct" the state of the site.
That's my only compliant about jupyter notebooks. It's a powerful tool but acts a little funky if you're not used to it.
Re: The Future of Notebooks: Lessons from JupyterCon
#36Earlier quoted context omitted.
My hunch is the same disdain you probably have towards scientists using excel is how many software engineers feel about your using jupyter. It isn't that you aren't getting good results and doing good work. It is that there are many practices that were hammered out in software development that were completely tossed out the window. 4GL comes to mind as a comparison. Makes great demos. Is not likely to lead to good en…
I use Jupyter for two things: 1. Interactive computation and plotting, run data through a pipeline. 2. Quick prototyping and testing. Once a function/class is ready, it goes into a module, together with some unit tests. Nothing beats Jupyter for that second use case, 90% of the bugs are squashed through interaction and inspection in the notebook cells, not through unit tests.
That some folks can be productive with a tool doesn't say, necessarily, much about the tool. Some could probably make the same two claims for excel. It isn't that the tool should be banned, per se. Just that many practices that have been rather proven in software are much harder to do in this environment.
Sounds like what you like is the live coding aspect. Many of the lisp environments of yesteryear would have probably appealed to you. So would many earlier math packages such as matlab and mathematica. Both of those have had environments similar to the new "notebooks" for many years.
For the pure coding side, imagine using an environment that actually allowed live redefinition of stuff in the code you had written. Step debugging. Breakpoints. Conditional breakpoints.
Re: The Future of Notebooks: Lessons from JupyterCon
#37I understand why they became popular, but as a software engineer considering how they work, I am just full of disappointment. we're going to spend the next ten years re-inventing every single software engineering best practice for jupyter's weirdo environment.
You reason from a downside (more layman programmers without proper workflow). I reason from an upside: much better flow than Excel or Excel/VBA and hence less errors, better accountability, etc.
Re: The Future of Notebooks: Lessons from JupyterCon
#38I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering.
So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE.
All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the problem of execution order that some people here mention). Then I may move those functions into a normal Python module and build up complex classes, add unit tests. Or if the goal is communicating with other people the results of running that code, I'll add explanatory text and mathematical formulas, break it up into sections, include references to papers and web links.
I think what people miss here is many software development tasks, especially in data science and machine learning, are essentially exploratory data analysis. And the outputs are graphs, tables of data etc, designed to be read by humans. Jupyter's ease of use for iteration, and the ability to interleave documentation, formulas, graphs and tables with the code is a big win.
In my daily work, I have my IDE and Jupyter open side by side. The Jupyter notebooks are version controlled like my .py modules.
Over time, I expect editors like VSCode to edit Jupyter notebooks natively. And also more code editing to move from IDEs into the Jupyter (via integrating the Monaco editor). We will all benefit from that crosspollination, whether we are principally software engineers or data scientists.
Re: The Future of Notebooks: Lessons from JupyterCon
#39I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…
> And also more code editing to move from IDEs into the Jupyter
Jupyter, and all REPLS in general, rely on a concrete top-level execution context to provide useful feeedback. Much of the code written outside of data science doesn’t really have that, which is why you don’t see notebooks being used for software development very often.
Re: The Future of Notebooks: Lessons from JupyterCon
#40Earlier quoted context omitted.
exactly. I use it for quick data exploration and experimentation but it remains at that level.
People do that in Excel too and the next thing you know a spreadsheet is managing a portfolio or being used as the basis for published science!
Its not the fault of the tools if users dont know any better.