Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notebooks Are McDonalds of Code

yobibyte.github.io

Post reply on HN
Print view View original

141–150 of 153 posts

Re: Notebooks Are McDonalds of Code

#141

Post by abdullahkhalids »

AlexanderTheGr8 wrote: post #84

The article and comments debating best practices in notebooks remind me of the time I hosted a website from a jupyter notebook. Technically, I was using gradio to create a localhost webpage and then piping it through cloudflare. The website would only work when the notebook was running on the cmd line of EC2. Hey if it works, it works! Notebooks allowed me to do a 2 week project in 2 hours. Like most things, notebook…

The online book I am writing is just a set of jupyter notebooks (with lots of code) that get converted to html and uploaded.

Re: Notebooks Are McDonalds of Code

#142

Post by esafak »

StableAlkyne wrote: post #90
esafak wrote: post #64

In the context of machine learning, notebooks are a crutch people lean on when the production environment lacks the right abstractions--isn't convenient enough. Notebooks fall under the "prototype smell" in Google's Hidden Technical Debt in Machine Learning Systems (2015). I agree that notebooks are best reserved for exposition.

I don't think I've met a single data scientist or ML person who used Python and didn't use notebooks. What are the "right abstractions" that everyone seems to miss?

I use them too, but for data science rather than machine learning. The right abstractions include handling authenticating to the ML services, experiment tracking, hyper-parameter optimization, distributed compute, access to PII data, etc. If you're dealing with nonsensitive data that fits on a laptop you might not need these.

Re: Notebooks Are McDonalds of Code

#143

Post by hot_gril »

lelanthran wrote: post #81
hot_gril wrote: post #52

Earlier quoted context omitted.

I agree with the quote, but this was code that we did modify frequently. It was still ok. 1600 line func is bad, but 1600 line file, well that's what editor splits are for. A lot of app backends don't do much other than shoving stuff into a DB and getting it back.

> I agree with the quote, but this was code that we did modify frequently. Could still be the best approach, if by "modify" you meant "added new queries to an array of strings".

We'd edit queries or add new HTTP endpoints. Some did a little more than query, maybe hit a cache or external service. There were a few shared helpers, not more than one layer deep though.

Re: Notebooks Are McDonalds of Code

#144

Post by [deleted] »

kelseyfrog wrote: post #71

It's absolutely hilarious that the author opens with an image of Socrates. Socrates, as you recall, famously argued that writing was a detriment to thinking. The parallel that notebooks are a sign of lazy thought does not go unnoticed.

[deleted]

Re: Notebooks Are McDonalds of Code

#145

Post by JoeyBananas »

kelseyfrog wrote: post #71

It's absolutely hilarious that the author opens with an image of Socrates. Socrates, as you recall, famously argued that writing was a detriment to thinking. The parallel that notebooks are a sign of lazy thought does not go unnoticed.

That is really not a valid parallel at all because code notebooks are only one particular form of coding.

Being against coding in a code notebook is more like saying "yellow highlighter on sticky notes is a bad medium for serious writing"

Re: Notebooks Are McDonalds of Code

#146

Post by kelseyfrog »

JoeyBananas wrote: post #145
kelseyfrog wrote: post #71

It's absolutely hilarious that the author opens with an image of Socrates. Socrates, as you recall, famously argued that writing was a detriment to thinking. The parallel that notebooks are a sign of lazy thought does not go unnoticed.

That is really not a valid parallel at all because code notebooks are only one particular form of coding. Being against coding in a code notebook is more like saying "yellow highlighter on sticky notes is a bad medium for serious writing"

This is a good opportunity to respond with the strongest plausible interpretation of what someone says.

Re: Notebooks Are McDonalds of Code

#147

Post by alan-hn »

rcxdude wrote: post #119
alan-hn wrote: post #58

Earlier quoted context omitted.

Sometimes I need to run my code in small pieces for testing and evaluation, I do this with multiple smaller scripts. Data can be saved to files and this accomplishes nearly the same thing as a notebook without needing to have the notebook environment

Sure, if you don't mind the overhead (both development and processing time) of loading/saving state to disk (about 10-30 minutes for a lot of my data). In notebooks you don't have to think about it since it's just the objects in memory (and indeed they don't make it easy to think about it, which is a reasonable criticism, but I don't currently know of a framework or system which gives you the advatages of both).

Whenever I work with large datasets I use a small subset of the overall data to do testing while I build the pipeline, this avoids long run times and allows for quick iteration while I get things set up to run against the full dataset

Re: Notebooks Are McDonalds of Code

#148

Post by squarepizza »

packetlost wrote: post #8

Some of the worst code I've seen in my life lives in Jupyter notebooks. But that's fine, it's meant to be throwaway code. The problem is a lot of places/people do not use it as such .

The insidiousness of notebooks is in their uncanny resemblance to proper code. To take writing for example, a handwritten draft scrawled on coffee-stained yellow paper is more obviously a work-in-progress than a typed manuscript. But the maturity of code is not so immediately apparent.

Re: Notebooks Are McDonalds of Code

#149

Post by skeledrew »

darepublic wrote: post #126
skeledrew wrote: post #76

Recently, since coming across FastAI and nbdev[0], I've been moving increasingly to a more notebook-centric flow. So far it's been better particularly for the exploration aspect where I've primarily used ptpython in the past (and this isn't anywhere ML-related). I think the idea behind nbdev is pretty neat, but it pushes some practices that I'm not a fan of at all. I want to get to the point where I have a mostly com…

I remember Jeremy from fast ai was an advocate for notebook centric flows. But I have to respectfully disagree that he is a good authority on maintainable / scalable coding practices like this. And though I may be wrong, I feel like the fast ai lib itself is a not too useful wrapper around pytorch and people would be better served just learning pytorch itself. I say this as someone who watched a year of the fast ai v…

I'd say the fastai library itself[0] is a pretty good example of how maintainable/scalable practices can come to life in notebook flows. There's something to be said IMO for an active project with 25.8k stars, 238 contributers, 2.7k commits, and 199 open vs 1.5k closed issues.

[0] https://github.com/fastai/fastai/

Re: Notebooks Are McDonalds of Code

#150

Post by int_19h »

lelandbatey wrote: post #55
Cacti wrote: post #9

Earlier quoted context omitted.

It’s a REPL, for starters.

REPL stands for Read–eval–print loop https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93prin... A REPL, by its name, is a very narrow version of the broader paradigm of interactive computer programming environments. But Notebooks are not REPLs, unless you use REPL to mean "interactive programming environment" and not REPL. Notebooks are much broader than a REPL! In a notebook, you can go back and edit and run in…

In modern parlance "REPL" means a lot more than that and typically incorporates at least some kind of editable history.
Post reply on HN