Why Jupyter is data scientists’ computational notebook of choice • Hacker News Board

Re: Why Jupyter is data scientists’ computational notebook of choice

#211

Post by joshuamorton » Tue, Oct 30, 2018, 9:59 PM UTC

jonnycomputer wrote: post #202

joshuamorton wrote: post #165

Earlier quoted context omitted.

As a result of the serialize-to-json approach, jupyter supports R, python, scala, go, lua, bash, julia, and haskell, among others. Its accessible to a much wider range of programmers, at the cost of version control being a bit weirder.

It might be the case that serializing to json facilitates support for multiple languages, though I wonder how. With the reticulate package in R Markdown you can run python chunks, by putting, e.g. ```{python} for i in range(1:10): print("{}:{}".format(i, i*i)) # etc ``` And in emacs org-mode you can: #+begin_src python for i in range(1:10): print("{}:{}".format(i, i*i)) #+end_src Language support in org-mode is prett…

Ah, actually it looks like I'm somewhat mistaken. R Markdown supports other languages as well. I think the real difference is that it doesn't look like R Markdown supports partial evaluation.

By that I mean that that to share the r-markdown doc it appears that you need to rerun the whole thing. It does some tricks to do concurrent visualization, but to actually share the doc you have to rerun all the R/python from scratch.

In jupyter OTOH, if I have a long running ML pipeline as part of my doc, I can render without rerunning the pipeline.

Re: Why Jupyter is data scientists’ computational notebook of choice

#212

Post by 3rdAccount » Tue, Oct 30, 2018, 10:02 PM UTC

celrod wrote: post #132

jonnycomputer wrote: post #93

Earlier quoted context omitted.

If you ever have used an R Notebook written in R-Markdown, then its pretty easy to see why Jupyter Notebooks putting everything in JSON is just... infuriatingly wrong-headed. In an R Notebook, I can see my code, I can see my text, everything is exceedingly simple to understand, and I can edit it in any of the fantastic text editors out there (Jupyter's editor is not among them)

RStudio is also my favorite editor. All my work is data science / stats related, where I like the workflow of writing/modifying code in a .R (or .py) file, and being able to quickly experiment by running chunks in a REPL with Ctrl + Enter. R and Python are supported. No Julia, unfortunately. VS Code and Atom support similar workflows with Julia. However, the Julia Language server in VS Code is extremely unstable and…

Is that atom's fault or just the Julia REPL's fault? I use the REPL directly on Windows and it seems to be really slow as it will take something like "using JuMP" and precompile the module which takes time.

Re: Why Jupyter is data scientists’ computational notebook of choice

#213

Post by narwally » Tue, Oct 30, 2018, 10:02 PM UTC

zmmmmm wrote: post #204

As with so many things python related (including python itself), I am perplexed by how willing people seem to be to fall in love with solutions that have so many limitations and problems. I find Jupyter just barely usable. I constantly have issues with editing in the cells, diagrams not sizing correctly, cells accidentally displaying huge amounts of data and freezing my browser, complete failure of autocompletion in…

I feel the same way, especially as an emacs user. Org-babel seems to be a superior implementation of the same idea. Org is just a text document, so git and git diffs work. I can use any combination of languages I want in a document and have them running in different sessions. And best of all I can edit code blocks using my customized major mode for that language. On top of that you get all the goodness that comes with org-mode, not least of which is the ability export it to dozens of other human readable formats for easy sharing. I think there's even an exporter for jupyter notebooks (there's at least one for ipython notebooks).

Re: Why Jupyter is data scientists’ computational notebook of choice

#214

Post by narwally » Tue, Oct 30, 2018, 10:05 PM UTC

gaius wrote: post #209

zmmmmm wrote: post #204

As with so many things python related (including python itself), I am perplexed by how willing people seem to be to fall in love with solutions that have so many limitations and problems. I find Jupyter just barely usable. I constantly have issues with editing in the cells, diagrams not sizing correctly, cells accidentally displaying huge amounts of data and freezing my browser, complete failure of autocompletion in…

it's a dumpster fire - in 2018 there should be something so much better than this In 1998 I was using a tool called MathCAD that provided a notebook interface running as a plugin to MS Word. In 2018, Jupyter is still not as good as that. Some things are just not meant to be webpages.

> Some things are just not meant to be webpages.

This is how I feel about most of the single page apps I've worked on.

Re: Why Jupyter is data scientists’ computational notebook of choice

#215

Post by 3rdAccount » Tue, Oct 30, 2018, 10:08 PM UTC

renjimen wrote: post #194

taeric wrote: post #81

Earlier quoted context omitted.

You parting "Granted..." is precisely what fills me with dread when I see notebooks. Yes, I have seen poorly done source files. I made more than a few myself. However, many of the practices we have grown into as sound programming advice seem to be largely thrown out the window for these notebooks. The irony, to me, is that I actually typically argue for the mixing of presentation and content. But to me, notebooks loo…

> Mathematica and Matlab both had "notebook" like features for a long long time. They probably didn't take off to the same extent as Jupyter because they're not free. IIRC MATLAB was quite expensive, particularly if you wanted to do anything specialised.

Yes, both are expensive outside the student licenses, but Mathematica is significantly cheaper and has a lot more built into the language, so you don't have to turn around and buy expensive "toolboxes" for all the functionality missing in Matlab.

Notebooks have been in Mathematica for ages and are really powerful and difficult to describe to those who haven't used them. To give an example, I was building a tool and embedded images as variables in a way reminiscent of being an engineer on the USS Enterprise. You can point to a file in Python as a variable, but you can't just copy-paste an image in as a variable last I checked (don't think Jupyter is there yet).

Re: Why Jupyter is data scientists’ computational notebook of choice

#216

Post by narwally » Tue, Oct 30, 2018, 10:10 PM UTC

makmanalp wrote: post #51

The majority of the complaints I hear about notebooks I think come from a misunderstanding of what they're supposed to be. It's a mashup between a scientific paper and a repl. So it's useful for a bit of both: a) Just like with a paper, you can present scientific or mathematical ideas with accompanying visualizations or simulations. From the REPL side, as a bonus, you get interactivity, and the reader can pause and e…

I'd say it's more of a shell than a REPL. For most languages that provide a shell, there isn't a real separation between the reader, evaluator, and the printer. Being able to interact with those components separately is the real advantage of a REPL over a shell.

Re: Why Jupyter is data scientists’ computational notebook of choice

#217

Post by narwally » Tue, Oct 30, 2018, 10:14 PM UTC

optimuspaul wrote: post #98

jonnycomputer wrote: post #93

Earlier quoted context omitted.

If you ever have used an R Notebook written in R-Markdown, then its pretty easy to see why Jupyter Notebooks putting everything in JSON is just... infuriatingly wrong-headed. In an R Notebook, I can see my code, I can see my text, everything is exceedingly simple to understand, and I can edit it in any of the fantastic text editors out there (Jupyter's editor is not among them)

YES! R Notebook is so much better in my opinion than Jupyter. I definitely prefer to work with a simple format verses JSON for this kind of work.

Agreed, but I'm coming from the org-mode side of things. Have you ever tried git diff on a JSON file? It's not always pretty.

Re: Why Jupyter is data scientists’ computational notebook of choice

#218

Post by red-tea » Tue, Oct 30, 2018, 10:19 PM UTC

zmmmmm wrote: post #204

As with so many things python related (including python itself), I am perplexed by how willing people seem to be to fall in love with solutions that have so many limitations and problems. I find Jupyter just barely usable. I constantly have issues with editing in the cells, diagrams not sizing correctly, cells accidentally displaying huge amounts of data and freezing my browser, complete failure of autocompletion in…

Why should there be something better? Just because you want it? Someone has to make it.

Re: Why Jupyter is data scientists’ computational notebook of choice

#219

Post by huac » Tue, Oct 30, 2018, 10:45 PM UTC

joshuamorton wrote: post #211

jonnycomputer wrote: post #202

Earlier quoted context omitted.

It might be the case that serializing to json facilitates support for multiple languages, though I wonder how. With the reticulate package in R Markdown you can run python chunks, by putting, e.g. ```{python} for i in range(1:10): print("{}:{}".format(i, i*i)) # etc ``` And in emacs org-mode you can: #+begin_src python for i in range(1:10): print("{}:{}".format(i, i*i)) #+end_src Language support in org-mode is prett…

Ah, actually it looks like I'm somewhat mistaken. R Markdown supports other languages as well. I think the real difference is that it doesn't look like R Markdown supports partial evaluation. By that I mean that that to share the r-markdown doc it appears that you need to rerun the whole thing. It does some tricks to do concurrent visualization, but to actually share the doc you have to rerun all the R/python from sc…

You can cache the results of an rmd cell, and you can also share the rendered version of the doc first. You're right that there's a higher emphasis on "run the whole thing," and I think that that's a conscious (and acceptable) design choice vs not being sure that the shared doc will run as provided.

Re: Why Jupyter is data scientists’ computational notebook of choice

#220

Post by taeric » Tue, Oct 30, 2018, 11:00 PM UTC

joshuamorton wrote: post #165

jonnycomputer wrote: post #93

Earlier quoted context omitted.

If you ever have used an R Notebook written in R-Markdown, then its pretty easy to see why Jupyter Notebooks putting everything in JSON is just... infuriatingly wrong-headed. In an R Notebook, I can see my code, I can see my text, everything is exceedingly simple to understand, and I can edit it in any of the fantastic text editors out there (Jupyter's editor is not among them)

As a result of the serialize-to-json approach, jupyter supports R, python, scala, go, lua, bash, julia, and haskell, among others. Its accessible to a much wider range of programmers, at the cost of version control being a bit weirder.

That is a complete non-sequitur. Json in no way enables that. Just having a defined format enables that.

Emacs org-mode is proof that a simple text format with markup rules is all you really need to support multiple languages in a single file. You lose some of the simplicity of parsing the file, but you gain a ton more.