Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Future of Notebooks: Lessons from JupyterCon

willcrichton.net

Post reply on HN
Print view View original

81–90 of 158 posts

Re: The Future of Notebooks: Lessons from JupyterCon

#81

Post by stewbrew »

TeMPOraL wrote: post #57
stewbrew wrote: post #34

Earlier quoted context omitted.

Some simpler notebook-like environments stay closer to source code in that they basically are source code with interleaved results (e.g. as comments). IMHO they hit a sweet spot between REPLs and those notebook environments inspired by mathematica, maple and similar more mathematically oriented software products.

That's just a REPL with basic editor integration (eval at point, paste result). Surprisingly unpopular outside Emacs/Lisp land.

Well yes, but these notebooks are little more, aren't they.

Re: The Future of Notebooks: Lessons from JupyterCon

#82

Post by jdkram »

stevesimmons wrote: post #79
Ixio wrote: post #66

Earlier quoted context omitted.

> The Jupyter notebooks are version controlled like my .py modules. That's interesting, could you give us some more details ? Last time I tried to put a Jupyter notebook under git it was as mess. Do you use another tool than git or have they made tools to help with version control ? Or is it just your workflow that helps, like emptying all cell results before saving ?

Our version control uses a monorepo, with non-branching workflows, development against the head version, and frequent commits/pushes to production. For Jupyter notebooks, cell results get emptied before saving, unless declared public. That is to ensure data confidentiality, not really to help version control. Of course, standard diff gets confused with ipynbs. You need a tool like nbdime (notebook diff and merge).

Thanks for sharing nbdime! Been looking for something like this for a while.

Re: The Future of Notebooks: Lessons from JupyterCon

#83

Post by peatmoss »

dangirsh wrote: post #42
zmmmmm wrote: post #9

Earlier quoted context omitted.

This is definitely one of my concerns too. Ad hoc code inside these notebooks is almost completely unmanageable from any reasonable software maintenance perspective, and refactoring code out of them is prohibitively difficult as well. I really want something to emerge that combines the best of both worlds of an IDE and notebook development, but there isn't anything close currently.

ob-ipython enables IDE-like editing features within the code cells. It's embedded in a polyglot, git-friendly, literate programming environment called Org-mode. I use it every day and love it. Other goodies: - easily manage multiple kernels (in different languages / machines) in one file - tree-based organization manages complexity better than linear notebooks - no browser in sight (unless you need interactive widget…

Org-mode is strictly speaking the most powerful notebook programming environment out there.

I’m also much more excited by R Notebooks as implemented by RStudio than I am Jupyter. R Notebooks take the same basic approach as org-mode, implementing a smaller set of functionality built around a more mainstream-palatable Markdown format.

I don’t like R Notebooks as much as org-mode (why reinvent the wheel!?), but at least the general approach plays nicely with git.

On the other hand, the lisp-addled part of me starts to think that the juypyter format being json might actually be a step up from a poorly specified markup format that has to be parsed into data structures.

Maybe the real issue is that git is an insufficient revision control system, and that we need revision control systems that can revision data structures, rather than simple text diffing.

Re: The Future of Notebooks: Lessons from JupyterCon

#84

Post by rmetzler »

jorgemf wrote: post #52
stevesimmons wrote: post #38

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…

Jupyter notebooks are for data science, mostly because visualization is required. Something you do once, report it and it is done. Itsn't make sense to use Jupyter for other stuff. It doesn't make much sense to use them for training big models of deep learning because there are better tools for that.

What tools can you recommend for machine learning?

Re: The Future of Notebooks: Lessons from JupyterCon

#85

Post by blattimwind »

stevesimmons wrote: post #38

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…

> So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE.

This approach to development might be a lot rarer than you think – I've never seen anyone do it frequently. For tiniest snippets ("If I want to split there and then use that suffix--how do I need to index? Let's check in the REPL real quick" or "What did that stdlib function return again? [Because that's of course not documented, why would anyone need to know that, after all]" and stuff like "I have this list of tuples like that, now, how do I need to zip/unpack/star that stuff such that I get a list of X instead. Let's try in the REPL with [(1,2), (3,4)]!"), sure, for "actual code" that's going somewhere... nope.

Re: The Future of Notebooks: Lessons from JupyterCon

#87

Post by gumby »

I found this most exciting:

> While Jupyter notebooks have traditionally been a humans-only entrypoint into a program, researchers and companies alike are increasingly using notebooks for automation.

It's part of a slow journey back the power of the Lispms and the Smalltalk environments.

Re: The Future of Notebooks: Lessons from JupyterCon

#88

Post by mi_lk »

stevesimmons wrote: post #38

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…

> Over time, I expect editors like VSCode to edit Jupyter notebooks natively.

Please, please, please. Given the rising importance of interactive programming, I really think it deserves more than a extension to give us a seamless experience.

Re: The Future of Notebooks: Lessons from JupyterCon

#89

Post by ur-whale »

"On the education side of things, Jupyter is quickly gaining adoption in universities around America, particularly for data science courses. Conversely, data science is increasingly becoming students’ first exposure to programming and computer science"

Am I the only one this two sentences worries deeply?

Or is it just a rehash of the twenty year old "my first exposure to programming was Excel" (which, from what I'm reading is what the article is talking about : a glorified spreadsheet).

Re: The Future of Notebooks: Lessons from JupyterCon

#90

Post by jaipilot747 »

peatmoss wrote: post #83
dangirsh wrote: post #42

Earlier quoted context omitted.

ob-ipython enables IDE-like editing features within the code cells. It's embedded in a polyglot, git-friendly, literate programming environment called Org-mode. I use it every day and love it. Other goodies: - easily manage multiple kernels (in different languages / machines) in one file - tree-based organization manages complexity better than linear notebooks - no browser in sight (unless you need interactive widget…

Org-mode is strictly speaking the most powerful notebook programming environment out there. I’m also much more excited by R Notebooks as implemented by RStudio than I am Jupyter. R Notebooks take the same basic approach as org-mode, implementing a smaller set of functionality built around a more mainstream-palatable Markdown format. I don’t like R Notebooks as much as org-mode (why reinvent the wheel!?), but at least…

Could you please elaborate on how you use orgmode for interactive programming?
Post reply on HN