Nbdev: A literate programming environment that democratizes best practices • Hacker News Board

Re: Nbdev: A literate programming environment that democratizes best practices

#31

Post by kowlo » Fri, Nov 20, 2020, 8:12 PM UTC

jph00 wrote: post #27

kowlo wrote: post #17

Earlier quoted context omitted.

I'm aware, what I'm confused about is why they've phrased it as if it's something they call exploratory programming, as if they've coined the term.

I coined the term - and it turns out someone else did too, for something else. So be it. If someone else can think of a better term that's never been used before, then I'll happily use that instead. The earlier usage mentioned in Wikipedia is entirely uncited there however, and seems to have only been used in one academic project AFAICT.

How are you sure you didn't just read it somewhere and then forgot? It has been mentioned many times in related literature (even in publication titles) https://scholar.google.com/scholar?q=%22exploratory+programm...

Here's one from 1988 https://dl.acm.org/doi/abs/10.1145/51607.51614

Are these unrelated? Is Nbdev not only a "new programming environment", but also a new concept that needs a new name?

Re: Nbdev: A literate programming environment that democratizes best practices

#32

Post by fulafel » Fri, Nov 20, 2020, 8:13 PM UTC

So you can put these in a GH action to ensure they are in working order across deps / dataset updates etc? Seems like exctly the missing piece for using notebooks for serious work.

Re: Nbdev: A literate programming environment that democratizes best practices

#33

Post by davidkell » Fri, Nov 20, 2020, 8:17 PM UTC

IMO no one has done more to make deep learning accessible than Jeremy + fast.ai team. Thanks for the amazing work!

My question is about the coding style - @jph00 I’ve read your fast.ai style guide and worked with APLs like q/KDB (written by Arthur Whitney who you cite).

My experience is that brevity is great, until you need to collaborate or have individuals working on small parts. That was my experience as well trying to write an extension to the fast.ai code (where I had to read large amounts of source to understand how to implement a small change).

Given that a key motivator for literate programming is collaboration/communication, how do you think about this?

Re: Nbdev: A literate programming environment that democratizes best practices

#34

Post by st1x7 » Fri, Nov 20, 2020, 8:18 PM UTC

sbelskie wrote: post #29

jph00 wrote: post #25

Earlier quoted context omitted.

No that's not the reason - or at least, not for everyone. I created nbdev. I've been coding for over 30 years, and spent over 10 years using R and S-PLUS. I've used Delphi, Visual Studio, Emacs, vim, vscode, and many other editors and IDEs, including many with integrated line-oriented REPLs. There's a big difference between a line-oriented plain text REPL, and a Mathematica/Jupyter-style notebook REPL, especially whe…

Yea, I don't really get what repls have to do with what notebooks offer. They are similar-ish, obviously, but they accommodate different workflows and use cases.

That's getting to my point though - they should accommodate different workflows and use cases. But what's happening instead is that people are overusing notebooks in cases where plain text + REPL is more appropriate.

Re: Nbdev: A literate programming environment that democratizes best practices

#35

Post by davidkell » Fri, Nov 20, 2020, 8:20 PM UTC

minimaxir wrote: post #19

st1x7 wrote: post #18

Sigh. I really think that the only reason notebooks are so popular is that Python never had a popular IDE with a high-quality REPL where people can learn to work interactively while writing code in plain text. R illustrates this very well - as an R user you can have your pick between RStudio, Jupyter and Rmarkdown and the overwhelming majority of users pick RStudio and notebooks are reserved only for a niche set of u…

You bring up RStudio but not the R Notebooks which it supports natively ( https://bookdown.org/yihui/rmarkdown/notebook.html ), which IMO is a far-superior way of handling notebooks than Jupyter. (namely, the files are plain text so you can actually commit to Git without fuss) I wrote a detailed blog post about the differences between Jupyter and R Notebooks years ago: https://minimaxir.com/2017/06/r-notebooks/

You can do the same thing with the jupytext extension. But sometimes it is helpful to have the rendered results in version control, eg internally we use it to discuss data science findings on Gitlab.

Re: Nbdev: A literate programming environment that democratizes best practices

#36

Post by hansvm » Fri, Nov 20, 2020, 8:20 PM UTC

jph00 wrote: post #28

hansvm wrote: post #7

Earlier quoted context omitted.

Guessing at plausible answers: - Nbdev started right around the 1.0 release of jupyterlab, and it might not have been on their radar - Nbdev came out of fast.ai, and I wouldn't be surprised if they were using a ton of jupyter-specific features already which weren't supported by jupyterlab

It works fine in lab too, although I prefer using notebooks on the whole. (I'm the lead author of nbdev).

Awesome! Thanks for chiming in :)

Re: Nbdev: A literate programming environment that democratizes best practices

#37

Post by slightwinder » Fri, Nov 20, 2020, 8:46 PM UTC

st1x7 wrote: post #18

Sigh. I really think that the only reason notebooks are so popular is that Python never had a popular IDE with a high-quality REPL where people can learn to work interactively while writing code in plain text. R illustrates this very well - as an R user you can have your pick between RStudio, Jupyter and Rmarkdown and the overwhelming majority of users pick RStudio and notebooks are reserved only for a niche set of u…

> Sigh. I really think that the only reason notebooks are so popular is that Python never had a popular IDE with a high-quality REPL where people can learn to work interactively while writing code in plain text.

I would not call Idle unpopular. But it had a limit for growing for sure.

But Notebooks are mostly coming from the science-corner of python. People there used notebook-like tools and workflows for decades and some brought that over to ipython-project. I remember 15(?) Years ago when the project started focusing more and more on the cluster-aspect of their shell, they brought up many differenct tools for this. One of them was notebook-like and what become later Jupyter. It quickly became popular in certain groups for those reasons.

Re: Nbdev: A literate programming environment that democratizes best practices

#38

Post by sbelskie » Fri, Nov 20, 2020, 8:47 PM UTC

st1x7 wrote: post #34

sbelskie wrote: post #29

Earlier quoted context omitted.

Yea, I don't really get what repls have to do with what notebooks offer. They are similar-ish, obviously, but they accommodate different workflows and use cases.

That's getting to my point though - they should accommodate different workflows and use cases. But what's happening instead is that people are overusing notebooks in cases where plain text + REPL is more appropriate.

That makes sense as a possible situation. I’m just not sure that it’s one that’s familiar to me. I can’t honestly say I have my finger on the pulse of where people are using notebooks vs repls but the former seem really great for 1) step by step examples 2) scripts-in-progress where certain steps are more in flux than others.

Though, I think I get what you mean as I reflect on my own dev experience. As a mainly C# dev which has a very very limited repl experience (I would and should say no repl experience but someone will yell at me about csi.exe or dotnet-script) I have seen people using notebooks for want of a good repl, but I’m curious why anyone writing python would.

Re: Nbdev: A literate programming environment that democratizes best practices

#39

Post by polyrand » Fri, Nov 20, 2020, 8:55 PM UTC

I used nbdev when it was first released. Some things must have improved since then, but I was already amazed by the experience. I think having code + docs + tests in the same document makes a huge difference in the effort needed to get those 3 done properly.

Re: Nbdev: A literate programming environment that democratizes best practices

#40

Post by fulafel » Fri, Nov 20, 2020, 8:58 PM UTC

sbelskie wrote: post #29

jph00 wrote: post #25

Earlier quoted context omitted.

No that's not the reason - or at least, not for everyone. I created nbdev. I've been coding for over 30 years, and spent over 10 years using R and S-PLUS. I've used Delphi, Visual Studio, Emacs, vim, vscode, and many other editors and IDEs, including many with integrated line-oriented REPLs. There's a big difference between a line-oriented plain text REPL, and a Mathematica/Jupyter-style notebook REPL, especially whe…

Yea, I don't really get what repls have to do with what notebooks offer. They are similar-ish, obviously, but they accommodate different workflows and use cases.

People mean different things by REPL - the nicer Lisps had richer reader prompts that were not totally text bsaed, could show you graphical stuff and accept commands other than just source code, have interactive features etc - see eg https://upload.wikimedia.org/wikipedia/commons/c/c6/Listener... or some youtube videos of Lisp machines.