What’s wrong with computational notebooks? • Hacker News Board

Re: What’s wrong with computational notebooks?

#221

Post by tgb » Thu, Jan 30, 2020, 12:11 PM UTC

ivirshup wrote: post #63

tgb wrote: post #47

Earlier quoted context omitted.

But, strangely, Jupyter doesn't also give you a REPL (like, say, R Studio does). I'm always making new cells in the middle to output the column names of my spreadsheet, and then I have to delete them. I used to just always have an ipython REPL running and test things out in there as I write. You can start a ipython instance on the same kernel but I found that messed up my plots when I did that IIRC.

You can get a REPL attached to a notebook in jupyter. When you open a console in jupyter-lab you have the option of attaching it to an already running kernel. Using the notebook interface you can connect a console using `jupyter console --existing`. By default this connects to the most recent session, but you can also specify a session by passing a token.

Yeah, but problems with that are what my last sentence referred to. I didn't try debugging it for long, though.

Re: What’s wrong with computational notebooks?

#222

Post by roblabla » Thu, Jan 30, 2020, 8:04 PM UTC

ivalm wrote: post #22

roblabla wrote: post #21

Earlier quoted context omitted.

Assuming the data isn't changed, thanks to CoW forking wouldn't cause any extra memory usage. If only a subset of data is changed, same thing - only the changed cells will take extra space. The problem only occurs when the whole variable changes - in which case yeah, you're SOL. I wonder what the usage patterns are for such datasets?

Personal experience: when first looking at the data I often do lots of map /reduce style operations which might transform large portions of the dataframe. Question, if you use CoW then presumably your variable blocks are no longer contiguous, wouldn't this really slow down vector operations?

> Question, if you use CoW then presumably your variable blocks are no longer contiguous, wouldn't this really slow down vector operations?

I don't think so. Vector operations require the data to be aligned to whatever the vector size is, no? E.G. 16-byte vector ops require the data to be aligned to 16-byte, etc... At least that's my understanding.

Re: What’s wrong with computational notebooks?

#223

Post by anchpop » Fri, Jan 31, 2020, 6:31 PM UTC

anonsivalley652 wrote: post #85

anchpop wrote: post #83

Earlier quoted context omitted.

> Maybe there should be standard, versioned interpreters for each kind of programming language in WebAssembly and cached for offline usage by the browser for inclusion in documentation, papers, etc. This would be incredible. Even better, the output from the code (like graphs) should be able to be embedded in the paper. You have no idea how many papers have errors in the code that generated the graphs/statistics/etc.…

For WASM, there ought to be a package-management/registry mechanism for installation (unless there is already? It might get complicated, but would seem a good idea to reuse code/plugins.)... or as below, there ought to be some caching priority mechanism. Then for HTML assets (and CSS ones too), perhaps a hint on asset-linking tags (a, script, link, img, audio, video, etc.) there ought to an offline-priority attribute…

Here's an idea - what if you could put a `hash` attribute on an script tag. After downloading whatever it links to, the browser checks the hash matches the one you provided. Then it could also cache the result and reuse it whenever the hash matches, even if the link pointed elsewhere.