101–110 of 223 posts
What are the currently available CI options for notebooks? You'd think this would be one of the first tools people would need to make sure notebooks are reproducible, but there seems to be little sign of CI usage.
Disclaimer: I wrote large part of treon.
1. They are stored by default in stupid json files instead of plain source code with comments.
2. The text editing interface inside the browser is horrific and very difficult to normalize (e.g., disable "smart" closing of parentheses, disable the capture of classic unix copy-pasting, etc).
I want a notebook where causality can only flow forward through the cells. I hate notebook time-loops where a variable from a deleted cell can still be in scope. 1. Checkpoint the interpreter state after every cell execution. 2. If I edit a cell, roll back to the previous checkpoint and let execution follow from there. I can't tell you how many times I've seen accidental persistence of dead state waste hours of peopl…
A couple years out of college we finally took a hard look at the credit cards and realized we had fucked up. We were gonna buckle down, pay the cards down hard for a while, 'color' our money so we both had discretionary spending separate from, say, the power bill. She had much more Excel experience than I did so she worked up a spreadsheet. It was bad. We had worked up some 'fair' notion of proportionality and she ba…
This is a really interesting insight (actually obvious when you think about it). I'm currently working on a spreadsheet app and these kinds of observations are very interesting to me. I guess things like named cells/variables will help (instead of using $A$4 etc.). Range selection could also be more intelligent (it could actively warn you if a range selection seems to be missing a few cells of the same data type). Do you have any other insights here?
Why no Mathematica?
Earlier quoted context omitted.
Doing data science in an IDE would be terrible. With a notebook, you get the chance to load the data, view it, clean it where needed, view it again, analyze it, model it and do anything else you need to it. An IDE means that you can't use the previous output to guide your next operation in a direct fashion like you can with a notebook.
as a counterpoint, plenty of R folks are pretty happy doing all of that in Rstudio
Earlier quoted context omitted.
My problem with notebooks is that I feel like the natural mental model for them is a spreadsheet mental model, not a REPL mental model. Under that assumption, changing a calculation in the middle means that all of the cells that depend on that calculation would be updated, but instead you need to go and manually re-run the cells after it that depend on that calculation (or re-run the entire notebook) to see the effec…
I keep seeing the complaint about state and I am getting sick of it. JUST HIT "RESTART KERNEL AND RERUN ALL CELLS," HOW HARD CAN IT BE?
Earlier quoted context omitted.
My problem with notebooks is that I feel like the natural mental model for them is a spreadsheet mental model, not a REPL mental model. Under that assumption, changing a calculation in the middle means that all of the cells that depend on that calculation would be updated, but instead you need to go and manually re-run the cells after it that depend on that calculation (or re-run the entire notebook) to see the effec…
I keep seeing the complaint about state and I am getting sick of it. JUST HIT "RESTART KERNEL AND RERUN ALL CELLS," HOW HARD CAN IT BE?
When something odd begins to happen, they don't immediately consider the possibility that it's not their bug and waste time trying to 'debug' the problem instead of just rerunning the notebook.
Earlier quoted context omitted.
My problem with notebooks is that I feel like the natural mental model for them is a spreadsheet mental model, not a REPL mental model. Under that assumption, changing a calculation in the middle means that all of the cells that depend on that calculation would be updated, but instead you need to go and manually re-run the cells after it that depend on that calculation (or re-run the entire notebook) to see the effec…
I keep seeing the complaint about state and I am getting sick of it. JUST HIT "RESTART KERNEL AND RERUN ALL CELLS," HOW HARD CAN IT BE?
Typically the time-intensive data prep stage is section 1.
The remaining sections are designed essentially like function blocks: data inputs listed in the first cell and data outputs/visualizations towards the end.
Once I decide the exploratory analysis in a section is more-or-less right, I bundle up the code cells into a standalone function, ready for reuse later in my analysis.
Jupyter notebooks can easily get disorganised with out-of-order state. However that is their strength too: exploratory analysis and trying different code approaches is inherently a creative rather than a linear activity.
I don't get why anyone one who knows how to use an IDE would ever use a notebook, the coding experience is garbage in comparison. I understand they started as a way to get STEM kids coding quick, but now they are like a standard in data analysis and data science, with those people needing experienced devs to translate the notebook into production code. This just drives the silo walls up higher.
Doing data science in an IDE would be terrible. With a notebook, you get the chance to load the data, view it, clean it where needed, view it again, analyze it, model it and do anything else you need to it. An IDE means that you can't use the previous output to guide your next operation in a direct fashion like you can with a notebook.
In a good data-oriented IDE like RStudio you get to do all of those things and write code which can be saved as plain text and can be version controlled well under git which you can't do well with Jupyter.
R folks have to be the best indicator in this case because they have access to a good IDE and they have good support for Jupyter. Their use is overwhelmingly in plain text files in RStudio, a small portion of rmarkdown notebooks and pretty much no one user R in Jupyter.
I don't get why anyone one who knows how to use an IDE would ever use a notebook, the coding experience is garbage in comparison. I understand they started as a way to get STEM kids coding quick, but now they are like a standard in data analysis and data science, with those people needing experienced devs to translate the notebook into production code. This just drives the silo walls up higher.
The Python IDEs for data science are mostly garbage - if you have any recommendations, I'm all ears because I really don't like notebooks but still keep switching between jupyter and vscode depending on what I'm working on.