Did I miss somewhere in the article where the author says what the chosen alternative is to notebooks? "I'm also constantly exploring other options and having fun" makes it sound like there isn't one.
Notebooks Are McDonalds of Code
31–40 of 153 posts
Re: Notebooks Are McDonalds of Code
#32I've never been a fan of notebooks, it always felt like a strange way to run my code. What's the real advantage over creating scripts? If I want my code to run somewhere I would need to notebook architecture or just turn it into a normal script anyhow
of course, running a script from start to finish is better when it's fast, but if your code is slow to run, each cell acts like an in memory cache of your previous work.
There's so many other ways to achieve the same effect though. I would argue that serializing the result of some expensive computation explicitly is even just better, as it can have a meaningful name and metadata.
Re: Notebooks Are McDonalds of Code
#33> If people are given a chance to be lazy, they will be (I will be for sure). Quickly hacking some stuff without properly testing? Sure! Linting? Pfff, living on the edge, you'll throw the colab away in an hour anyways
To save everyone time, I think the fundamental disagreement comes down to this. A vast chunk of the audience of notebooks is people whose brain has not been drilled on the daily about best practices and maintainable code and often don't get judged on that criteria or affected by the consequences of this. If you believe that these people are irredeemable as engineers, or more charitably that the incentives driving this cannot be fixed, then you probably believe in taking away good tools from these people because they'll misuse it, and then you'll agree with these articles. If you don't, then you won't.
> I am using code autoreload and write code in modules >> Nice! I've done this for a while as well. This is a good use-case. However, this approach does not address some of the issues, e.g. data analysis scripts should be checked by another team member.
How come? When you write your code in a module, it gets committed and PR'd and linted and tested and reviewed just like any other code, which is the whole point of this approach. Which I think is the most reasonable and mature one.
The notebook then is mostly reserved for a very thin interaction layer optimized for play, on top of your vast iceberg of reviewed & tested & maintainable code. Including lots of prewritten convenience routines for plotting and munging to minimize business logic in the notebooks further, and not have paragraphs of matplotlib crud in hidden cells. You construct what's almost like a shell + DSL that's optimized for the domain and datasets you care about, and you can focus on investigating. If there's more significant code, it soon goes in a module again, without interrupting your ongoing flow and thought process.
Re: Notebooks Are McDonalds of Code
#34Notebooks are spreadsheets of code
No, I assure you, even in 2024, there are definitely fields where spreadsheets are still the spreadsheets of code.
Re: Notebooks Are McDonalds of Code
#35Earlier quoted context omitted.
You can't embed graphs in a script, and plotting is an important part of systematic research. Also, it is easier to have an obvious sequential set of experiments over months in a notebook rather than a bunch of scripts. It's the same reason scientists use lab notebooks to keep track of things rather than just a bunch of loose papers.
Yes, I have a lab notebook and I work in a lab. However I use folder structures to organize my scripts and as for graph outputs I save them as files. In my line of work I generate hundreds of graphs for data analysis verification and a notebook isn't set up to handle things like that as far as I can tell
My workflow normally starts in the shell (SQL, R/rush/ggplot csvtk, etc), moves to Jupyter (F# in .NET Interactive, with some SQL and R/ggplot inlined), and then to a Makefile when I’m ready to make some deliverables.
I’ve got an ever growing tool set of F# data related functions that I’ve moved to a personal lib in my dotfiles and use in scripts, notebooks, etc.
Re: Notebooks Are McDonalds of Code
#36I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.
Some people, such as Jeremy Howard, think otherwise:
Re: Notebooks Are McDonalds of Code
#37The article should start with more context, what is a notebook ? I know what it is, but the author is particularly bad at introducing his article.
Agreed, even something like "coding" notebooks if brandname usage is a concern would help. It took more time than I'd like to admit to understand that they weren't talking about physical notebooks...
Re: Notebooks Are McDonalds of Code
#38Some of the worst code I've seen in my life lives in Jupyter notebooks. But that's fine, it's meant to be throwaway code. The problem is a lot of places/people do not use it as such .
Idk how you'd use one for non-throwaway code. Not like a webserver can run a routine in a notebook.
Re: Notebooks Are McDonalds of Code
#39Re: Notebooks Are McDonalds of Code
#40Some of the worst code I've seen in my life lives in Jupyter notebooks. But that's fine, it's meant to be throwaway code. The problem is a lot of places/people do not use it as such .
In general anything written by nonprofessional programmers in pursuit of some other goal is terrible code by professional programmer standards. On the other hand, it accomplishes a goal other than getting a programmer paid. So in one important sense is objectively better than probably 80% of the code I've seen people paid to write, no matter how nicely it was constructed.
Amen to that. The confluence of an insulator nature of organisational developer cliques, the culture of self-importance in the field, and the ability for many teams to expend material effort on things that are…at best, tangentially related to business goals (due to cheap money, and frankly, often pulling the wool over management’a eyes), has bred at least a couple of generations of developer where large contingents have a completely out of whack perspective on what’s actually important.