Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notebooks Are McDonalds of Code

yobibyte.github.io

Post reply on HN
Print view View original

81–90 of 153 posts

Re: Notebooks Are McDonalds of Code

#81

Post by lelanthran »

hot_gril wrote: post #52
giraffe_lady wrote: post #50

Earlier quoted context omitted.

Yeah I think I got this from Sandi Metz a long time ago but it's been so valuable to me: "bad code that you never have to modify is good code." Now when business requirements change and you're constantly sending people into that 1600 line backend function or whatever, it's likely worth it to start doing some refactors. But otherwise if I don't have a good reason to be in there I pretend I don't see it.

I agree with the quote, but this was code that we did modify frequently. It was still ok. 1600 line func is bad, but 1600 line file, well that's what editor splits are for. A lot of app backends don't do much other than shoving stuff into a DB and getting it back.

> I agree with the quote, but this was code that we did modify frequently.

Could still be the best approach, if by "modify" you meant "added new queries to an array of strings".

Re: Notebooks Are McDonalds of Code

#82

Post by fifilura »

waldrews wrote: post #30

Notebooks are a diluted form of the Lisp/Smalltalk REPL-based development experience, with some features from the reactive-spreadsheet world. Especially for those of us in the business of producing numbers, 'real code' with a fixed set of tests isn't a better way but a necessary evil, a black box that we can't really trust. Building a calculation piece by piece, in a notebook, trying out variations along the way? Tha…

This is what I have to teach people working with numbers. Often with a risk of looking like a fool to the SWE crowd.

That working with numbers is a craft in itself. And the primary driver for the craft are the numbers. Not SWE practices.

You can't "feel" the numbers just by coming up with a huge testsuite.

And pretty often, the feel goes missing when you translate your prototype notebook into "real code".

Re: Notebooks Are McDonalds of Code

#84

Post by AlexanderTheGr8 »

The article and comments debating best practices in notebooks remind me of the time I hosted a website from a jupyter notebook.

Technically, I was using gradio to create a localhost webpage and then piping it through cloudflare. The website would only work when the notebook was running on the cmd line of EC2. Hey if it works, it works! Notebooks allowed me to do a 2 week project in 2 hours.

Like most things, notebooks have their pros and cons. One of the biggest adv is very rapid experimentation. Even a regular script takes a while for python interpreter to run and that time (even if only a few seconds) adds up in lack of creativity ("Bret Victor - Inventing on Principle" [1]). And if your script is loading a big database, then notebook is a no-brainer.

One of the biggest disadv of notebook is mis-ordering. You are allowed to declare a variable in cell 3 and then go and use it in cell 2. Even worse, you can declare a variable in cell 3 and then delete cell 3 and still be able to use that variable. That I believe is the biggest dis-adv of notebooks. It adds way too many subtle errors. One way to bypass this is to write everything in functions - no global vars.

I am willing to accept the issue of mis-ordering in order to get rapid experimentation. It's subjective whether you think the pros outweight the cons. I definitely think they do.

[1] : https://www.youtube.com/watch?v=PUv66718DII&t=4s

Re: Notebooks Are McDonalds of Code

#85

Post by xanathar »

lucw wrote: post #13

The article should start with more context, what is a notebook ? I know what it is, but the author is particularly bad at introducing his article.

Especially given that 'notebook' is also a synonym of 'laptop' and I was like, wtf, I don't have a mainframe to be less lazy.

Re: Notebooks Are McDonalds of Code

#86

Post by fifilura »

analog31 wrote: post #74
vineyardlabs wrote: post #10

I'm an embedded person, so clueless on this stuff. Are notebooks really used in prod? Outside of applications where notebooks are specifically useful to the goal of the system, ie. documentation,blog posts,textbooks with interactive/data driven elements? How would this even work? Can an ipynb file be mechanized in the same way a py file can?

I think the issue actually goes beyond notebooks. It's just that notebooks are the method du jour for creating hacked-together code. But regardless of the tool chosen, hacked-together code does eventually leak into production. Mine has. It's harder to breach that barrier in embedded, because someone has to approve of getting the hardware made. In most organizations, there's a kind of no mans land between hacked code,…

"and career wise -- should be working on the biggest project"

That explains a lot. I have seen a lot of SWE that believe they are working on bigger project but they are not. Probably 90% of them.

These are the guys that will produce a lot of useless abstractions without producing any real value.

I guess they aspire to work with kafka/k8s at Google but ended up at the local shop.

And nothing wrong with the local shop. This is probably the place where you actually have the possibility to produce real value.

Re: Notebooks Are McDonalds of Code

#87

Post by waldrews »

skydhash wrote: post #77
waldrews wrote: post #30

Notebooks are a diluted form of the Lisp/Smalltalk REPL-based development experience, with some features from the reactive-spreadsheet world. Especially for those of us in the business of producing numbers, 'real code' with a fixed set of tests isn't a better way but a necessary evil, a black box that we can't really trust. Building a calculation piece by piece, in a notebook, trying out variations along the way? Tha…

I wonder why CL and Smalltalk haven’t beat Python. Is is the languages or just unawareness? The workflow just make more sense there with better updates propagation and state saving.

The Clojure/JVM statistics/scientific computing/now tensor math packages just never got as good as Python, and in Smalltalk they were a non-starter. R is an awkward language, but it's repl-first, has a lot of Lisp's metaprogramming (done in very ad-hoc ways), and Smalltalk's serializable image model -- so a lot of exploratory/experimental statistical methods research happens there, and then gradually makes its way to Python when it needs to be stabilized for production.

F# based .Net would've made a fine math-centric environment, but it's a language with a high initial barrier, and though .Net community has been making a decent effort at porting over a lot of NumPy/SciPy, it's not fully caught up after many years.

We have Julia now, it's jitted, multicore/GPU friendly, and has interesting REPL innovations, and yes, serializable state (though, ugh, ligatures). But every time I reach for it, it's like, oh no, yet another thing I want to call is in the Python ecosystem. Especially all the modern deep learning/tensor stuff, where the assumption is Python's speed and the GIL don't matter because you're just gluing together GPU calls.

Re: Notebooks Are McDonalds of Code

#88

Post by vundercind »

alan-hn wrote: post #5
jhbadger wrote: post #3

I think people may be misusing notebooks. They aren't there to develop software but serve as virtual versions of scientific notebooks (hence the name). They are there to conduct experiments (changing parameters and the like in your code) and to record and plot the results. You don't have to develop the software itself in the notebook.

I can change parameters in a script. What's the advantage?

They’re a repl where you can go back and edit and re-run earlier parts much more easily than on a normal repl.

Re: Notebooks Are McDonalds of Code

#89

Post by casualscience »

> Notebooks are often used as a playground to easily have access to an accelerator. In this case, the execution environment is often set up differently and with different dependencies. In this case, you make your code work in a colab, smile widely and run your experiment after on a cluster or whatever. You start crying in twenty minutes after your code crashes due to lack of dependency or versioning or anything else similar.

This is actually one of the best things about jupyter is that you can just tunnel into the computer and use the exact same env... this article is WACK

Re: Notebooks Are McDonalds of Code

#90

Post by StableAlkyne »

esafak wrote: post #64

In the context of machine learning, notebooks are a crutch people lean on when the production environment lacks the right abstractions--isn't convenient enough. Notebooks fall under the "prototype smell" in Google's Hidden Technical Debt in Machine Learning Systems (2015). I agree that notebooks are best reserved for exposition.

I don't think I've met a single data scientist or ML person who used Python and didn't use notebooks.

What are the "right abstractions" that everyone seems to miss?

Post reply on HN