Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What’s wrong with computational notebooks?

web.eecs.utk.edu

Post reply on HN
Print view View original

151–160 of 223 posts

Re: What’s wrong with computational notebooks?

#151

Post by CaptArmchair »

azhenley wrote: post #10

Co-author of the study here. Let me know if you have any questions or how you overcome some of the problems we identified!

Question for the author:

Have you looked into the domain of "research data management"? Concerns such as "archival", "security" or "share & collaborate" are core to this research domain:

In academics, there's a trend to prepare a "data management plan" up front that creates awareness about these concerns. They are even a requirement in order to get funding:

i.e. https://dmponline.dcc.ac.uk/

So, it's a bit odd to see a study that's focussed on a single technical tool yield the same concerns... but not making that jump to a larger, existing framework on information management.

Looking at the authors, it seems you are located at Oregon State University. A quick DuckDuckGo search yields this service from your colleagues at the University Library:

https://guides.library.oregonstate.edu/dmp

With the context of notebooks themselves, I think the study reflects on "if you have hammer, every problem looks like a nail." Notebooks aren't the only powerful tool to work with data. I think many of the same concerns could be raised with Google Sheets or Excel with heavy VBA scripting. Like others said, this is not a new problem.

Notebooks do have a place in the bigger process of doing iterative research based on data mining techniques. They can help to formulate more accurate questions and perform quick tests without the friction of having to set up complex environments. Moving on from initial data exploration, it's up to the researcher to use a formal method and tools that do mitigate those concerns. RDM is all about providing tools and mitigating (legal) liabilities as far as "what do you do with your data?" is concerned.

Re: What’s wrong with computational notebooks?

#152

Post by tardenoisean »

https://datalore.io has (1) a reactive Datalore kernel that solves the reproducibility problem. It recalculates the code automatically when something is changed, and recalculates only the changed and the dependent code; (2) good completion; (3) online collaboration; (4) read-only sharing; (5) publishing; (6) sensitive data can be saved in .private directory that is not exposed when the notebook is shared with read-only access

Re: What’s wrong with computational notebooks?

#153

Post by tardenoisean »

moultano wrote: post #5

I want a notebook where causality can only flow forward through the cells. I hate notebook time-loops where a variable from a deleted cell can still be in scope. 1. Checkpoint the interpreter state after every cell execution. 2. If I edit a cell, roll back to the previous checkpoint and let execution follow from there. I can't tell you how many times I've seen accidental persistence of dead state waste hours of peopl…

https://datalore.io does exactly this

Re: What’s wrong with computational notebooks?

#154

Post by PeterisP »

archi42 wrote: post #115

As a computer scientist/software engineer, please allow me the question: Why would I prefer a notebook over e.g. equivalent python script(s) in a git? I first saw jupyter notebooks when my sister (physicist, non-programmer) used it for analyzing economical data with pandas. Run-time for the full data set was half a day (and IMHO for that analysis SQL would have been better suited). I understand that as a non-programm…

The key factor is iteration speed.

If step A takes 5 minutes (and 5 minutes is a very short time) and I want to experiment on step B, then I don't want to rerun step A each time while I'm writing and running code that helps me understand what step B is going to be; I'd want that to be interactive and immediate, not have each rerun take 5 minutes.

Storing/loading to disk is not a good option because all the data that needs to be stored is not yet determined until the exploration is finished; If I write code to save/load A, then I need to change (and test) it after I'm done with B and now want to experiment with C, and it all becomes even more complicated when I need to add an extra step and data field to step A and rerun everything. Deciding what data should be stored in what format is something that you can do in 'productionizing' the code after you've done the exploratory analysis.

REPL is not a good option because it's not convenient to save and replicate the code that got you to the current REPL state.

The other aspect is that visually 'debugging' intermediary data through various plots is not conveniently possible in IDEs. I could generate some picture files in a folder or possibly an HTML 'dashboard' to see the results of my most recent run but that takes extra code and effort, and the results aren't immediately in my face like in a notebook.

Re: What’s wrong with computational notebooks?

#155

Post by SiempreViernes »

jacobolus wrote: post #40
WorldMaker wrote: post #31

Earlier quoted context omitted.

That's something that I think Observable [1], in my modest usage, seems to do well. [1] https://observablehq.com/

Yep, the real complaint is “dead state”, not out of order execution. Worrying about linear flow per se turns out to be misguided based on lack of imagination for/experience with a better model: reactive re-rendering of dependent cells. Observable entirely solves the dead state problem, in a much more effective way than just guaranteeing linear flow would do. * * * More generally, Observable solves or at least amelior…

You can get a jupyter extension[1] that allows you to add tags and dependencies and this way construct the dependency graph as you go along. Of course, you have to do it manually and the interface is a bit clunky, but it does what it says.

In practice I think taking care not to accidentally shadow variables is much more important: this dependency business only makes sense once you have a clear idea of what you need and by that point you are mostly done anyway.

[1] https://jupyter-contrib-nbextensions.readthedocs.io/en/lates...

Re: What’s wrong with computational notebooks?

#156

Post by SiempreViernes »

tgb wrote: post #47
sillysaurusx wrote: post #37

Earlier quoted context omitted.

It's best to think of the notebook as a REPL. So you'd want to run `del foo` on the old name. In fact, this is a good counterexample. Why should the notebook delete the old variable name? What if its value is a thread currently executing? Notebooks are REPLs, and it's better to get used to that than to try to enforce some confusing time traveling.

But, strangely, Jupyter doesn't also give you a REPL (like, say, R Studio does). I'm always making new cells in the middle to output the column names of my spreadsheet, and then I have to delete them. I used to just always have an ipython REPL running and test things out in there as I write. You can start a ipython instance on the same kernel but I found that messed up my plots when I did that IIRC.

Just get the extension: https://jupyter-contrib-nbextensions.readthedocs.io/en/lates...

there are quite a few very useful ones, my favourite being collapsible headings: https://jupyter-contrib-nbextensions.readthedocs.io/en/lates...

Re: What’s wrong with computational notebooks?

#157

Post by LeanderK »

tardenoisean wrote: post #152

https://datalore.io has (1) a reactive Datalore kernel that solves the reproducibility problem. It recalculates the code automatically when something is changed, and recalculates only the changed and the dependent code; (2) good completion; (3) online collaboration; (4) read-only sharing; (5) publishing; (6) sensitive data can be saved in .private directory that is not exposed when the notebook is shared with read-on…

it seems it's cloud-based. Fun for playing around but not suitable for real work (at least for me).

I can't just upload random data to some cloud service to work with it, also I can't upload data if it's too big. Often the data that's valuable is very sensitive.

Re: What’s wrong with computational notebooks?

#158

Post by nestorD »

tastyminerals wrote: post #130
nestorD wrote: post #116

Earlier quoted context omitted.

Yes! Rstudio is the one thing I miss most when doing datascience in python. Notebooks give me some of the interactivity but the experience degrades significantly. The spyder IDE seem like an okayish replacement but some of the library I use expect you to have html display (within a notebook) to give you full functionalities which is not yet available in spyder.

Have you tried Orange. It has scripting capabilities.

No but, looking at some screenshot + descriptions, it seems to get me further from the code which does not seem like what I am looking for

Rstudio gives you the experience of a classical IDE + easy data exploration which I found to be productive from the exploratory stages (where I need to see my data and the effect of my code) to the clean-up phase (where I refactor my file).

Re: What’s wrong with computational notebooks?

#159

Post by kriro »

I'm curious. How do people protocol their experiments? When I started, I used to just keep the cells but that lead to very long and impossible to parse Jupyter notebooks. I have since opted for keeping a journal.txt file in Atom where I write down hpyerparameter configurations, epochs run and results (for ML). But that feels a bit awkward as well.

Re: What’s wrong with computational notebooks?

#160

Post by serendipityisme »

sunaden wrote: post #114

I work at https://www.deepnote.com/ , we are trying to tackle some of the pains mentioned in the article (setup, collaboration, IDE features like auto-complete or linting). We are still early access, but if you are interested in an invite just let me know. My email is filip at deepnote dot com.

Deepnote seems quite interesting, but as a cheapskate grad student, I'm compelled to ask.

If this information isn't private, what sort of business model do you use? I take it you'll have a SaaS subscription model? I see it's free to use now, but how does your company plan to make money (especially taking into account the cost of the cloud hosting Deepnote requires)?

Post reply on HN