[deleted]
Notebooks Are McDonalds of Code
21–30 of 153 posts
Re: Notebooks Are McDonalds of Code
#22Earlier quoted context omitted.
You can't embed graphs in a script, and plotting is an important part of systematic research. Also, it is easier to have an obvious sequential set of experiments over months in a notebook rather than a bunch of scripts. It's the same reason scientists use lab notebooks to keep track of things rather than just a bunch of loose papers.
Yes, I have a lab notebook and I work in a lab. However I use folder structures to organize my scripts and as for graph outputs I save them as files. In my line of work I generate hundreds of graphs for data analysis verification and a notebook isn't set up to handle things like that as far as I can tell
You are asking what the advantage of a notebook is. Can you truly not see a job / workflow in which inline plotting would be advantageous? I find that very hard to believe.
Re: Notebooks Are McDonalds of Code
#23The article should start with more context, what is a notebook ? I know what it is, but the author is particularly bad at introducing his article.
Indeed. I have to admit it took me some times to understand this was not about a specific small-ish form factor of laptops (e.g. Chromebooks like)…
Re: Notebooks Are McDonalds of Code
#24I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.
I can change parameters in a script. What's the advantage?
Remember back university, we used to do course projects (for science/engineering course) and at the end we would have to write a report. The report would typically include, among other stuff, two things.
(1) The calculational methods we used - could either be a set of mathematical equations or a description of the algorithms. (2) The results of evaluating these equations/algorithms for different parameter values. Usually some graphs, and some discussion of their meaning.
A Jupyter notebook is designed to replicate that process but make it easier because the figures are produced by code right there. Personally, all my notebooks include a discussion in the markdown cells what I am doing, and why. It includes discussions of the code. And directly from the code, some graphs or numbers, with a discussion attached.
With the script workflow, I would have two different files. One with the code, and one with the results pasted in. It's annoying when my primary goal is to develop and test the algorithms under discussions. Best thing is, if done right, my work is completely replicable. Just run the notebook again.
Just because some people misuse the tool doesn't mean the tool isn't useful.
Re: Notebooks Are McDonalds of Code
#25Some of the worst code I've seen in my life lives in Jupyter notebooks. But that's fine, it's meant to be throwaway code. The problem is a lot of places/people do not use it as such .
Idk how you'd use one for non-throwaway code. Not like a webserver can run a routine in a notebook.
Re: Notebooks Are McDonalds of Code
#26A notebook is a REPL with an inline wiki. Of course it is not intended for running production code, it’s just an R&D environment to document, share, and test ideas
Re: Notebooks Are McDonalds of Code
#27I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.
I can change parameters in a script. What's the advantage?
Sometimes you want to quickly iterate on a portion of code doing some sort of analysis or tweaking a plot. If your data pipeline takes a while to run then rerunning the whole script is really awful. Notebooks make it easy to cache rerun and tweak chunks of the code.
Re: Notebooks Are McDonalds of Code
#28I've never been a fan of notebooks, it always felt like a strange way to run my code. What's the real advantage over creating scripts? If I want my code to run somewhere I would need to notebook architecture or just turn it into a normal script anyhow
Notebooks have at least one advantage, but it's one that shouldn't really be an advantage: In a world where every big organisation rushed to adopt all the latest cloud buzzwords, we now have an awful lot of half-baked fontends for AWS and Azure that will certainly be mandated by company policy. Actually getting a computer that can access all the data you want and run in a way you want through EU law, US law, EULAs, company security policy, cloud billing policy et cetera et cetera is a nightmare.
But for some reason, no matter what nonsensical environment you're being forced to operate in, there's always a Jupyter web interface with access to exactly what you need that none of the auditors, accountants or systems administrators seem to care about. It may not be good, but it works, and won't be bricked by some inexplicable change to IAM roles or VPS settings applied by some automated policy tool you didn't even know exists.
Re: Notebooks Are McDonalds of Code
#29I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.
I can change parameters in a script. What's the advantage?
Another advantage is when you have very slow code, you can use cells as caches, essentially without having to worry about serialization to disk. This often makes it much easier to interactively explore/develop downstream methods without needing to re-run earlier upstream dependencies.
This is especially useful with large datasets. Even if serialization is straightforward, if you have enough data (or the data is remotely hosted), loading it might take anywhere from 2s to multiple minutes, and even 2s is enough to get you out of the flow if you are working rapidly and want quick feedback.
Re: Notebooks Are McDonalds of Code
#30Notebooks are a diluted form of the Lisp/Smalltalk REPL-based development experience, with some features from the reactive-spreadsheet world. Especially for those of us in the business of producing numbers, 'real code' with a fixed set of tests isn't a better way but a necessary evil, a black box that we can't really trust. Building a calculation piece by piece, in a notebook, trying out variations along the way? That's how you get a feel for the calculation, connect with it at a spiritual level.