Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JupyterCon: I don't like Notebooks [slides]

docs.google.com

Post reply on HN
Print view View original

101–110 of 112 posts

Re: JupyterCon: I don't like Notebooks [slides]

#101

Post by reilly3000 »

wting wrote: post #97
jules wrote: post #95

What if notebooks re-executed all cells in order as you type? That would solve the ordering and hidden state problem. To speed that up you could take a snapshot of the program state at each cell and re-execute from the snapshot of the cell preceding the cell you're modifying. The advantage of a notebook over a repl is that the code you typed stays there and can be re-run and modified later. Re-executing all cells in…

This causes issues when working with large data sets or running expensive operations, which is an advantage of modifying a single step independently from the rest.

As an example, loading up a dataset I was working with into RAM from my NVME drive took about 9 minutes each time. Rerunning that step on each script run is productivity-prohibitive as I am just trying to explore and clean the data.

Re: JupyterCon: I don't like Notebooks [slides]

#102

Post by jules »

wting wrote: post #97
jules wrote: post #95

What if notebooks re-executed all cells in order as you type? That would solve the ordering and hidden state problem. To speed that up you could take a snapshot of the program state at each cell and re-execute from the snapshot of the cell preceding the cell you're modifying. The advantage of a notebook over a repl is that the code you typed stays there and can be re-run and modified later. Re-executing all cells in…

This causes issues when working with large data sets or running expensive operations, which is an advantage of modifying a single step independently from the rest.

Copy on write handles most cases fine. You can also still use such a system as a repl: just add more lines and don't modify previous lines. If you do that then nothing needs to be re-executed.

Since most people work with small data sets or can use a subset of their dataset during development, I think this would be a better default for a notebook.

Re: JupyterCon: I don't like Notebooks [slides]

#103

Post by phonon »

wting wrote: post #97
jules wrote: post #95

What if notebooks re-executed all cells in order as you type? That would solve the ordering and hidden state problem. To speed that up you could take a snapshot of the program state at each cell and re-execute from the snapshot of the cell preceding the cell you're modifying. The advantage of a notebook over a repl is that the code you typed stays there and can be re-run and modified later. Re-executing all cells in…

This causes issues when working with large data sets or running expensive operations, which is an advantage of modifying a single step independently from the rest.

http://www.pgbovine.net/incpy.html

Re: JupyterCon: I don't like Notebooks [slides]

#104

Post by Dowwie »

FridgeSeal wrote: post #24

I seriously cannot agree more. Jupyter notebooks were fun to use for a bit, then I hit the inevitable wall of "ok, now let's turn this into a real, properly built script, but now everything is breaking for inexplicable reasons". Notebooks are fine for early stage experimenting, but if you've got to the point where you start up you are relying on a notebook for anything, or your workflow consists of "start up notebook…

One-off "experiments" become production technology all the time. They address an immediate need at the cost of tomorrow. This hasn't ever changed. Hyperbolic discounting is real.

When you're in a job that requires running experiments, analyzing results, and summarizing findings, all within an hour, shortcuts are required.

I think that analysts/scientists using python and Jupyter notebooks are doing IT a favor. If your job is to refactor/re-engineer python notebooks and corresponding workflows, you've either forgotten how much worse it could be with excel+vba or you fortunately arrived after trench warfare.

Re: JupyterCon: I don't like Notebooks [slides]

#105

Post by megamindbrian2 »

megamindbrian2 wrote: post #70
FridgeSeal wrote: post #24

I seriously cannot agree more. Jupyter notebooks were fun to use for a bit, then I hit the inevitable wall of "ok, now let's turn this into a real, properly built script, but now everything is breaking for inexplicable reasons". Notebooks are fine for early stage experimenting, but if you've got to the point where you start up you are relying on a notebook for anything, or your workflow consists of "start up notebook…

I wrote a loader for this. And there are tools to split cells for compiling. I wrote a tool to split up cells too and compile in order. Pretty easy, all the code is in notebooks.

A few quick searches, maybe something along these lines? https://jupyter-notebook.readthedocs.io/en/stable/examples/N...

My JavaScript version I just put a simple wrapper around my print-results call to check if it is in module mode or notebook mode.

Re: JupyterCon: I don't like Notebooks [slides]

#106

Post by sireat »

Notebooks are a fantastic teaching tool despite some valid criticisms in the slides.

I teach Python to adult beginners including those with heavy Excel background.

Beginners catch on quickly that you need to run all the cells once to get reproducible results.

Worst comes to worse you just restart the kernel.

With Anaconda I can start teaching immediately over all 3 platforms(Win,MacOS and there are even Linux Python beginners) without worrying about differing versions or troubleshooting installations.

My students just git clone/pull my projects and start following around and experimenting themselves.

What would be the alternative, give them black on white REPL?

That said for personal projects that I do not intend to share I reach for my trusty text editor(VS Code is pretty good these days).

Re: JupyterCon: I don't like Notebooks [slides]

#108

Post by sstanie »

kovek wrote: post #87
daveguy wrote: post #73

Earlier quoted context omitted.

I don't even like it for initial experimentation and exploration. Much prefer the command line repl (the jupyter/ipython one). Faster feedback and easier navigation then you can just export the history when you're ready to capture and make permanent some workflow. It still requires going through and picking out the important bits, but reading the history like a story of commands is generally enough to pick out the im…

Will the REPL allow you to easily edit function definitions and then reinterpret them?

Yep, in the ipython REPL if you do

  %load_ext autoreload
  autoreload 2
it'll reload functions from any module you edit each time you run a new command.

Re: JupyterCon: I don't like Notebooks [slides]

#109

Post by hatmatrix »

kbumsik wrote: post #57
hatmatrix wrote: post #54

Earlier quoted context omitted.

Exactly - it's a classic case of a tool being used for a different purpose ("IDE") than its original intention ("presentation software") and getting criticized for not holding up to misguided expectations. His criticism is valid but the main lesson is to use the appropriate tool for the job. He himself uses VSCode + iPython, which is a better tool more generally for the desired use case of code development.

Although the OP criticizes missing IDE features but that is not his main points.

But his main points aren't really showstoppers if you're using it as presentation software.
Post reply on HN