71–80 of 308 posts
I love notebooks as a way to present information, data, code and computations. However, I cannot stand typing any text into a web browser window. Is there any way to edit a jupyter notebook with a text editor and then run it in the browser? The native json is not really human-editable.
Earlier quoted context omitted.
When you're processing a lot of data, it can be expensive to keep re-running your whole script every time you make a change. The notebook keeps the results of your earlier steps in memory when you want to change and re-run a later step. This is a trade-off between how much code you're writing and how much data you're processing. If you're writing maybe 20 lines of code but you have enough input that it takes several…
You can generally persist the results your self to disk though. Especially since a lot of things end up being numpy arrays. So you run 1 script that saves all the results, and another that loads it and runs just the part of your workflow you want. Bonus: it's persisted to disk on top of that! I know things get more complicated than that, but I'd say the compelling use case for notebooks isn't the state saving but mor…
Installing R packages through anaconda is like pulling teeth and the docker images for my Jupyter notebooks push past 6GB and take multiple cups of tea to build.
Is there a good solution I'm missing? A good hosted solution perhaps?
RStudio with using Rmarkdown is also popular. Both workflows are language agnostic.
Putting aside the R vs Python question (as as noted in this thread, you can use R in a Jupyter notebook and Python in an RMarkdown notebook), I much prefer RMarkdown notebooks. RMarkdown notebooks are plain text, so you can read them easily in any text editor (which also means they play well with git, unlike Jupyter notebooks). And it's meant to work with the RStudio IDE, so I get a much more seamless experience goin…
Notebooks are great for invoking existing functions and exploring data. Notebooks aren't ideal for creating functions (standard text editor features are lacking and testing is impossible). Notebooks encourage an "order dependent variable assignment" programming style without abstractions. Here's what you'll commonly see in a notebook: val df = spark.read.csv("some_data") df2 = df.withColumn("clean_name", trim("name")…
[1] Github: https://github.com/janushendersonassetallocation/loman [2] Quickstart/Docs: https://loman.readthedocs.io/en/latest/user/quickstart.html
I like R for many things, but Python just keeps getting more compelling, particularly given the excellent machine learning packages. As these sorts of toolchain elements get better and better, and as more people realize that there's a benefit to simultaneously training researchers to run code as well as stats, I suspect we'll start to see an exodus from pure R solutions. The real question is when (and whether) new so…
> and as more people realize that there's a benefit to simultaneously training researchers to run code as well as stats, I suspect we'll start to see an exodus from pure R solutions
I'm not sure what you are saying here.
I would actually argue that most of the Python data science toolchain is years behind what is available in R.
Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/
Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/
I think coupling to github makes sense if you are a building a dev-support service, but for a end user it makes little sense to wed the vcs to a specific website.
Version control for Jupyter notebooks was one of the biggest complaint I had. Specifically, diff and merge with the JSON files (.ipynb) is ugly. I built ReviewNb[1] to solve one of those problems (diff). Note that, there is nbdime[2] which works well for local diff/merge. The idea for ReviewNb is to have much tighter integration with GitHub etc. [1] https://reviewnb.com [2] https://nbdime.readthedocs.io/en/latest/
I kind of find Jupyter an indictment of other coding tools really, it's 2018 and they're normally kind of weak or kind of unprogrammable. Feel like we're waiting for someone to really reinvent Emacs, preferably using web tech. Most editors can't open a terminal that you can use VIM keybindings on to search/navigate history and treate like any other buffer. VSCode -> not currently possible because they wrote it in a r…