Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Jupyter is data scientists’ computational notebook of choice

nature.com

Post reply on HN
Print view View original

181–190 of 308 posts

Re: Why Jupyter is data scientists’ computational notebook of choice

#181

Post by huac »

ChuckMcM wrote: post #172

I see Jupyter notebooks as the next step in spreadsheets with more code foundation and different media support. For me the interface doesn't work as well as I would like but I can see the potential. My brother in law wants something like this for structural analysis reports where the code, data and report are all one thing that can be pulled out and examined. Watching the new iPad announcement today I think this is s…

yeah, i have a use case now where we pull data from a database, manipulate it, and then have a final table/csv/dataframe/whatever. the problem is then how to share this with non-technical users. in an ideal world, this would get inserted into a google sheet, and that sheet would just update daily after new data is loaded into the database.

i'm pretty sure this is a usecase which others have and curious what people use to solve it. i've heard, variously, that some options are to use tableau/similar or email a csv and ask the end user to import into google sheets/excel

Re: Why Jupyter is data scientists’ computational notebook of choice

#182

Post by huac »

azag0 wrote: post #164

I love Jupyter Notebook for experimenting and rapid creation of reports, but dislike it for not being able to use my editor and for intermingling inputs and outputs in a single file. So I'm working on an alternative frontend to Jupyter kernels, which is heavily inspired by KnitR: https://github.com/azag0/knitj It is still being developed, but it's functional and I use it every day.

your project also seems to play much more nicely with git!

Re: Why Jupyter is data scientists’ computational notebook of choice

#183

Post by smortaz »

thomasfedb wrote: post #73

Jupyter is lovely (and JupyterLab looks delicious ), but the set up required to achieve a reproducible local server with the R kernel and versioned R packages is 100% not. Installing R packages through anaconda is like pulling teeth and the docker images for my Jupyter notebooks push past 6GB and take multiple cups of tea to build. Is there a good solution I'm missing? A good hosted solution perhaps?

https://notebooks.azure.com (python, r, f#)

Re: Why Jupyter is data scientists’ computational notebook of choice

#184

Post by TeMPOraL »

zimablue wrote: post #99
gnulinux wrote: post #80

Earlier quoted context omitted.

I use Jupyter in emacs: ein-mode. The whole concept of programming in a browser sounds bizarre to me. I have a tool that's designed for programming (emacs) and a tool that's designed for streaming cat videos (firefox) and I use the latter for programming? Thanks but no thanks. Ime emacs works pretty perfect with jupyter too, there is no need to use firefox for something it's not designed to do for my full-time job.

I have also tried ein-mode, and sometimes use it. I haven't used it enough to have a super educated opinion but it didn't blow me away. I don't think it supports cell folding (?) as an example missing feature Minor point but it also can't/shouldn't support widgets, which we use at work. Any extension to jupyter is going to be written in javascript so there's an element I'd be locking myself out of the ecosystem. I di…

> I don't think it supports cell folding (?) as an example missing feature

If I understand what you mean correctly, that would be handled by built-in outline-minor-mode, or by a third-party Emacs module like yafolding or fold-this.el. Emacs packages tend to be made to compose well with other packages (it's a requirement given how everyone's Emacs is a special snowflake, unlike any other Emacs).

As for widgets/Jupyter extensions, then yes. Emacs can't really help you there AFAIK.

Re: Why Jupyter is data scientists’ computational notebook of choice

#185

Post by azag0 »

huac wrote: post #182
azag0 wrote: post #164

I love Jupyter Notebook for experimenting and rapid creation of reports, but dislike it for not being able to use my editor and for intermingling inputs and outputs in a single file. So I'm working on an alternative frontend to Jupyter kernels, which is heavily inspired by KnitR: https://github.com/azag0/knitj It is still being developed, but it's functional and I use it every day.

your project also seems to play much more nicely with git!

True. Actually that's what I meant by "intermingling inputs and outputs". KnitJ still shows both code and its output in the rendered HTML, but unlike in Jupyter Notebook, the code is stored and edited separately in a single source file.

Re: Why Jupyter is data scientists’ computational notebook of choice

#186

Post by y4mi »

azag0 wrote: post #164

I love Jupyter Notebook for experimenting and rapid creation of reports, but dislike it for not being able to use my editor and for intermingling inputs and outputs in a single file. So I'm working on an alternative frontend to Jupyter kernels, which is heavily inspired by KnitR: https://github.com/azag0/knitj It is still being developed, but it's functional and I use it every day.

> for not being able to use my editor

At least Atom has good integration with hydrogen

Re: Why Jupyter is data scientists’ computational notebook of choice

#187

Post by mikepurvis »

azag0 wrote: post #185
huac wrote: post #182

Earlier quoted context omitted.

your project also seems to play much more nicely with git!

True. Actually that's what I meant by "intermingling inputs and outputs". KnitJ still shows both code and its output in the rendered HTML, but unlike in Jupyter Notebook, the code is stored and edited separately in a single source file.

That's my biggest frustration by far with trying to use Jupyter for anything Serious Business. Will definitely check out your project; thanks for sharing.

Re: Why Jupyter is data scientists’ computational notebook of choice

#188

Post by mlevental »

celrod wrote: post #132
jonnycomputer wrote: post #93

Earlier quoted context omitted.

If you ever have used an R Notebook written in R-Markdown, then its pretty easy to see why Jupyter Notebooks putting everything in JSON is just... infuriatingly wrong-headed. In an R Notebook, I can see my code, I can see my text, everything is exceedingly simple to understand, and I can edit it in any of the fantastic text editors out there (Jupyter's editor is not among them)

RStudio is also my favorite editor. All my work is data science / stats related, where I like the workflow of writing/modifying code in a .R (or .py) file, and being able to quickly experiment by running chunks in a REPL with Ctrl + Enter. R and Python are supported. No Julia, unfortunately. VS Code and Atom support similar workflows with Julia. However, the Julia Language server in VS Code is extremely unstable and…

r studio supports python? does it completion and stuff like that?

Re: Why Jupyter is data scientists’ computational notebook of choice

#189

Post by tcpekin »

Having spent a decent amount of time learning to be a programmer while doing scientific image analysis in Matlab (shudders from the real programmers), and with a decent amount of time spent in Mathematica as well, I just can't seem to buy into the Jupyter/notebook based programming enthusiasm. The talk linked in the article explains it better than I ever could, but for me, when I am leaving data in memory, it is much more convenient for me to have a completely linear history, ordered by command execution time. In python I have found the best way to do this is writing standard python functions and scripts, and running them in an IPython environment with the %run magic. You have the linear history, git works well on standard .py files, and you can interactively work with the data in the IPython prompt without worrying that something is proceeding nonlinearly. What I find works best is to explore with the data in the live prompt, which gives you interactivity, and then build up slowly a master collection of functions and commands that when run, with a single command, can reproduce the results you got while exploring. Then to come back to the data at a later point in time, you have to run one script file on the raw data. Of course, this is kind of the point of the Jupyter notebook, but I find that often when I want to change parameters, the ability to jump around and redefine things means I do. By moving from IPython to a script/functions I run, I ensure that everything progresses linearly. Idk, just my two cents.

Re: Why Jupyter is data scientists’ computational notebook of choice

#190

Post by celrod »

mlevental wrote: post #188
celrod wrote: post #132

Earlier quoted context omitted.

RStudio is also my favorite editor. All my work is data science / stats related, where I like the workflow of writing/modifying code in a .R (or .py) file, and being able to quickly experiment by running chunks in a REPL with Ctrl + Enter. R and Python are supported. No Julia, unfortunately. VS Code and Atom support similar workflows with Julia. However, the Julia Language server in VS Code is extremely unstable and…

r studio supports python? does it completion and stuff like that?

It uses reticulate for REPL support, and has basic Python completions and snippets.

However, it doesn't seem to complete variable names. Eg, if I defined "foobar", it won't let me tab-complete that later when I start typing "foo".

Post reply on HN