Live data from Hacker News

Why Jupyter is data scientists’ computational notebook of choice

nature.com

161–170 of 308 posts

Re: Why Jupyter is data scientists’ computational notebook of choice

#161
post #87

Earlier quoted context omitted.

How do you see the idea of a vs code notebook comparing to or being different from the goals of the hydrogen editor?

To me, my favorite part of the design of Hydrogen is that it's entirely language agnostic, and can be used with _any_ Jupyter kernel.

I'm planning on adding better language support in a couple of weeks! Don't want to limited to Python and R

Re: Why Jupyter is data scientists’ computational notebook of choice

#162
post #87

Earlier quoted context omitted.

Hey There! I'm trying to solve the issue of IntelliSense.. I'm building/improving Jupyter Notebooks inside VSCode: https://github.com/pavanagrawal123/VSNotebooks . It's a fork from another extension somebody already built, but all activity is dead, so I'm starting up dev on an active fork. I'd love to hear any feedback y'all have! :) Also planning to add some nice debug features, plus hopefully integration into the i…

How do you see the idea of a vs code notebook comparing to or being different from the goals of the hydrogen editor?

Better debugging is something I want to focus on in VSNotebooks.

Personally, I like VSCode more than Atom, so this was one of the reasons I started working on this extension!

Re: Why Jupyter is data scientists’ computational notebook of choice

#163
Have people been finding that AWS SageMaker disconnects you and you have to restart after a couple hours?

I am curious people's thoughts on using Jupyter for long-running code. Having a totally self-contained experiment in one notebook, even if it long-running, is very useful for reproducibility. It works fine on my local laptop and a remote server, but not with SageMaker.

Re: Why Jupyter is data scientists’ computational notebook of choice

#164
I love Jupyter Notebook for experimenting and rapid creation of reports, but dislike it for not being able to use my editor and for intermingling inputs and outputs in a single file. So I'm working on an alternative frontend to Jupyter kernels, which is heavily inspired by KnitR: https://github.com/azag0/knitj It is still being developed, but it's functional and I use it every day.

Re: Why Jupyter is data scientists’ computational notebook of choice

#165

The majority of the complaints I hear about notebooks I think come from a misunderstanding of what they're supposed to be. It's a mashup between a scientific paper and a repl. So it's useful for a bit of both: a) Just like with a paper, you can present scientific or mathematical ideas with accompanying visualizations or simulations. From the REPL side, as a bonus, you get interactivity, and the reader can pause and e…

If you ever have used an R Notebook written in R-Markdown, then its pretty easy to see why Jupyter Notebooks putting everything in JSON is just... infuriatingly wrong-headed. In an R Notebook, I can see my code, I can see my text, everything is exceedingly simple to understand, and I can edit it in any of the fantastic text editors out there (Jupyter's editor is not among them)

As a result of the serialize-to-json approach, jupyter supports R, python, scala, go, lua, bash, julia, and haskell, among others. Its accessible to a much wider range of programmers, at the cost of version control being a bit weirder.

Re: Why Jupyter is data scientists’ computational notebook of choice

#166

Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/

[deleted]

Re: Why Jupyter is data scientists’ computational notebook of choice

#167

Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/

I nicknamed one we used during the Ebola epidemic (tight deadlines, lots of people working, etc.) "The Wall of Madness".

There were tons of

## JOHN: DONT RUN PAST HERE, EVERYTHING BROKEN

comments.

Re: Why Jupyter is data scientists’ computational notebook of choice

#168

Earlier quoted context omitted.

RStudio’s Markdown notebooks do not suffer from this and save a separate output file that can be gitignored.

And they pay for this on other accounts: No inline rendering of markdown. Opening an .Rmd file is a lottery to see if rendered graphs and tables still exists. Tables render completely differently in editor, HTML and pdf

For me, markdown is meant to be readable even when not rendered. I could see how not having persistent graphs and tables might be an issue, but my own philosophy is to start fresh each time - I treat it like a templating language with some convenient rendering features for prototyping, rather than like an IDE.

Your last point also has an upside - it's using different engines (Rmarkdown vs. Sweave). I can write whatever HTML or LaTeX code I want, depending on what's appropriate. I wouldn't want to have to make web documents with LaTeX, nor would I want to make PDFs with HTML.

Re: Why Jupyter is data scientists’ computational notebook of choice

#169

Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/

Tighter integration with git is very interesting, but this is sadly just integration with github. I think coupling to github makes sense if you are a building a dev-support service, but for a end user it makes little sense to wed the vcs to a specific website.

The RCloud project covered some of this ground https://cscheid.net/2015/08/17/collaborative-visual-analysis... It takes the view that everything should be saved and versioned. In hindsight it seems obvious that this can overwhelm people with dead ends and scratch work and in general the flat workbook space doesn't provide enough help with organizing results. There are some other ideas mentioned in the conclusion of the RCloud paper.

Re: Why Jupyter is data scientists’ computational notebook of choice

#170

Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/

The hard part is that introducing a tool like git (which requires you to choose moments to take a snapshot of the file, and then add some commit message) breaks the flow of interactive experimentation that notebooks are so good for. And then we need to find a way to make those commits useful, because the time ordering of commits could be different from the time order in which cells were run! That is what is crucial t…

I'm not a Jupyter user, but I solve the reproducibility problem with Make.

As a project moves from exploration toward production, the entire thing is wrapped into a Makefile that can flow from raw data to publication in a single call to make.

Post reply on HN