Live data from Hacker News

Why Jupyter is data scientists’ computational notebook of choice

nature.com

231–240 of 308 posts

Re: Why Jupyter is data scientists’ computational notebook of choice

#231

Earlier quoted context omitted.

Any chance of this being offered for on-prem install in the future? Looks interesting but cloud only makes it a no go for my team.

We are seriously considering such a possibility. Do you have any specific requirements for on prem installation?

Basically just the ability to run on Linux.

Re: Why Jupyter is data scientists’ computational notebook of choice

#232

I love notebooks as a way to present information, data, code and computations. However, I cannot stand typing any text into a web browser window. Is there any way to edit a jupyter notebook with a text editor and then run it in the browser? The native json is not really human-editable.

If you are comfortable writing python programs, you can use nbformat package (https://github.com/jupyter/nbformat) to generate Jupyter notebooks:

http://nbviewer.jupyter.org/gist/fperez/9716279

Re: Why Jupyter is data scientists’ computational notebook of choice

#233
post #204

As with so many things python related (including python itself), I am perplexed by how willing people seem to be to fall in love with solutions that have so many limitations and problems. I find Jupyter just barely usable. I constantly have issues with editing in the cells, diagrams not sizing correctly, cells accidentally displaying huge amounts of data and freezing my browser, complete failure of autocompletion in…

As soon as a git repo involves jupyter notebooks I move on. They're ugly, they don't let me learn how the code works properly and in general looks awful. Why not just give me some code to run??

Re: Why Jupyter is data scientists’ computational notebook of choice

#234
post #208

Does anybody know of a good hosted solution of JupyterHub? I made a neat notebook that I needed to share with my non-technical team, it was using iPyWidgets to do some interactive modeling, but they each needed to be able to use it independently. It has private data so I couldn't use Binder. I've been following Zepl.com for a long time, but couldn't use them here because Zepplin doesn't support iPyWidgets. Pretty soo…

https://the-littlest-jupyterhub.readthedocs.io/ is really easy to deploy on your own server!

Now this is my speed, thanks for sharing!

Re: Why Jupyter is data scientists’ computational notebook of choice

#235

I love notebooks as a way to present information, data, code and computations. However, I cannot stand typing any text into a web browser window. Is there any way to edit a jupyter notebook with a text editor and then run it in the browser? The native json is not really human-editable.

As a matter of fact, you can treat .ipynb notebook as data format (https://nbformat.readthedocs.io/en/latest/format_description...) and build systems that generate/consume nbformat data. Jupyter/JupyterLab are examples of nbformat generator/consumer. GitHub/Gist/nbviewer (http://nbviewer.jupyter.org/) are examples of nbformat consumer.

Re: Why Jupyter is data scientists’ computational notebook of choice

#236
post #58

Earlier quoted context omitted.

I hate to be the stereotypical Julia recommender, but it is made for this use case, more so than Python, which isn’t all that much faster than R if speed matters. (Unless you want to try Cython but that’s a whole bag of worms.)

I'd second that. R and Python both have the same pre-LLVM performance issues. I don't expect either R or Python to go away either time soon, nor would I want them to, but I would like to see people moving to things like Julia and Nim, which have the same level of expressivity, but are much more performant. I have difficulty imagining many people saying "I love programming in R and Python, but don't like Julia or Nim.…

I would also add two good reasons to stick with R: RStudio and Hadley Wickham.

In theory, there are Python and Julia equivalents to RStudio (JupyterLab, Spyder, PyCharm, Juno, whatever) but RStudio is just so, so, so good. A truly great piece of software.

And of course if you have a data pipeline type workflow, and it fits into the Hadleyverse paradigm and isn't too performance intensive, there's nothing better.

Re: Why Jupyter is data scientists’ computational notebook of choice

#237
post #226

Earlier quoted context omitted.

I feel the same way, especially as an emacs user. Org-babel seems to be a superior implementation of the same idea. Org is just a text document, so git and git diffs work. I can use any combination of languages I want in a document and have them running in different sessions. And best of all I can edit code blocks using my customized major mode for that language. On top of that you get all the goodness that comes wit…

I expect jupyter notebooks to keep an environment consistent between cells. Org-Babel doesn’t generally do this. A Good notebook environment is more like a lisp buffer with block comments.

Org Babel source blocks can take a session property to main consistent environment(s) across code blocks. The session property can also be set as a language specific file-level property eg: https://orgmode.org/manual/Header-arguments-in-Org-mode-prop...

Re: Why Jupyter is data scientists’ computational notebook of choice

#238
post #170

Earlier quoted context omitted.

The hard part is that introducing a tool like git (which requires you to choose moments to take a snapshot of the file, and then add some commit message) breaks the flow of interactive experimentation that notebooks are so good for. And then we need to find a way to make those commits useful, because the time ordering of commits could be different from the time order in which cells were run! That is what is crucial t…

I'm not a Jupyter user, but I solve the reproducibility problem with Make. As a project moves from exploration toward production, the entire thing is wrapped into a Makefile that can flow from raw data to publication in a single call to make.

This has been my solution as well. There’s little that feels as good as running `make -B report` and watching the whole thing be rebuilt from scratch.

How do you manage encapsulating each step, and passing data between them?

Re: Why Jupyter is data scientists’ computational notebook of choice

#239

The majority of the complaints I hear about notebooks I think come from a misunderstanding of what they're supposed to be. It's a mashup between a scientific paper and a repl. So it's useful for a bit of both: a) Just like with a paper, you can present scientific or mathematical ideas with accompanying visualizations or simulations. From the REPL side, as a bonus, you get interactivity, and the reader can pause and e…

> When I see stuff around notebooks for "reproducibility", I'm a bit confused in that notebooks often don't specify any guidance on installation and dependencies, let alone things like arguments and options that a regular old script would. At the core of this, as some others may have already alluded to already, is that many academic scientists have not been socialized to make a distinction between development and pro…

Huge fan of just including an environment.yml for a conda virtual-env in the repo you store your notebooks in, but the challenge there is that it's OS specific reproducibility. I've had no luck creating a single yml for all OS's and the overhead of creating similar yml's for (say) Mac and Win is a lot unless you plan on sharing your notebook widely.

Re: Why Jupyter is data scientists’ computational notebook of choice

#240
post #219

Earlier quoted context omitted.

Ah, actually it looks like I'm somewhat mistaken. R Markdown supports other languages as well. I think the real difference is that it doesn't look like R Markdown supports partial evaluation. By that I mean that that to share the r-markdown doc it appears that you need to rerun the whole thing. It does some tricks to do concurrent visualization, but to actually share the doc you have to rerun all the R/python from sc…

You can cache the results of an rmd cell, and you can also share the rendered version of the doc first. You're right that there's a higher emphasis on "run the whole thing," and I think that that's a conscious (and acceptable) design choice vs not being sure that the shared doc will run as provided.

Yeah I don't disagree about that, but when the question is why is this ugly format more popular, the answer "one has a share button and the other can be edited in emacs" sort of gives you the answer.
Post reply on HN