Nbdev: Create delightful software with Jupyter Notebooks
1–10 of 25 posts
Re: Nbdev: Create delightful software with Jupyter Notebooks
#2A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post:
https://www.fast.ai/2022/07/28/nbdev-v2/
Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
Re: Nbdev: Create delightful software with Jupyter Notebooks
#3A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post: https://www.fast.ai/2022/07/28/nbdev-v2/ Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
ive been a colab and deepnote uswr for a while, especially deepnote foe collaborative work.
how much is this focused on collaboration?
Re: Nbdev: Create delightful software with Jupyter Notebooks
#4A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post: https://www.fast.ai/2022/07/28/nbdev-v2/ Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
Thanks Jeremy et al for a great piece of software.
I have used nbdev numerous times to introduce data scientists to good practices in software development.
Getting them to go from zero to a well tested, documented, CI/CD-ready code in an hour and seeing their faces light up always brings me joy. Keep up the great work.
Re: Nbdev: Create delightful software with Jupyter Notebooks
#5I don't dislike notebooks, but I definitely have one foot in the "don't use notebooks for serious software" camp. I recently worked on a project where we're trying a notebook as the development platform for a data visualization report. The designer uses the notebook + bokeh to iterate on the report, and then we use nbconvert with some environment variables to create reports for different datasets.
My biggest issue with this paradigm is we actually had a lot of problem getting notebook development to work consistently bug-free on the various environments we have (Windows, Linux, VScode vs in-browser Jupyter, etc). It seemed like it would've been so much easier to just use a vanilla python script that generates the html report files. With hot reloading the iteration could be just as fast.
The other issue is that everything was horribly slow with the amount of data we were dealing with (~150MB of json). This is probably more related to python/bokeh than the notebooks themselves, but it meant that re-executing some cells was painful and would often hang or block the IDE.
Re: Nbdev: Create delightful software with Jupyter Notebooks
#6Hamel here, one of the core developers on this project. I just want to say that we are really excited about this new release of nbdev and the added functionality it brings to users. The thing I'm most excited about is that you can use nbdev for more things than ever before, such as: Documenting existing codebases (even if they aren't written in nbdev), blog posts, books, presentations, etc.
We also have an amazing notebook runner, as well as many other quality of life improvements. We will be adding more tutorials, walkthroughs and examples in the coming days. If you are interested in using nbdev please get in touch!
Other resources:
- A detailed walkthrough: https://www.youtube.com/watch?v=l7zS8Ld4_iA&t=3225s
- Blog post: https://www.fast.ai/2022/07/28/nbdev-v2/
- Documentation: https://nbdev.fast.ai/getting_started.html
- Conversation b/w Jeremy Howard and JJ Allaire: https://youtu.be/xxVVSxcjNQs
Re: Nbdev: Create delightful software with Jupyter Notebooks
#7A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post: https://www.fast.ai/2022/07/28/nbdev-v2/ Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
ive been a colab and deepnote uswr for a while, especially deepnote foe collaborative work. how much is this focused on collaboration?
It's not focused on collaboration, but it does add some critical pieces that otherwise make Jupyter development frustrating when working with a team. Specifically: `nbdev_prepare` ensures that diffs are as small as possible, by removing and standardising notebook metadata; and `nbdev_fix` fixes merge conflicts so that they are cell-level, rather than line level, so they can be opened and fixed in notebooks.
Something else we've found helpful for collaboration (not associated - just happy users) is this: https://www.reviewnb.com/ . It means we can get a nice notebook-based PR workflow.
Real-time collaboration is available in Jupyter nowadays: https://jupyterlab.readthedocs.io/en/stable/user/rtc.html . nbdev doesn't have any extra functionality for it, however -- but it should work fine in this environment.
Re: Nbdev: Create delightful software with Jupyter Notebooks
#8I don't dislike notebooks, but I definitely have one foot in the "don't use notebooks for serious software" camp. I recently worked on a project where we're trying a notebook as the development platform for a data visualization report. The designer uses the notebook + bokeh to iterate on the report, and then we use nbconvert with some environment variables to create reports for different datasets. My biggest issue wi…
I did run into some problems with nbconvert from time to time. It's worth noting that nbdev doesn't use nbconvert at all - it uses Quarto instead (which AFAICT does everything nbconvert does plus a lot more, and is faster, more extensible, and more "batteries included").
Having said that, it's possible that, given your experiences with needing to re-run some slow cells and having trouble making that work well, you might prefer to use "pure Quarto" instead of nbdev. With Quarto you can write your report as a .qmd file directly: https://quarto.org/ .
Personally, I quite like the notebook environment for situations like this where there are some really slow cells -- I mainly do deep learning, and some of my cells take many hours to run -- since that state is cached and I can easily manipulate it and visualise it afterwards. I generally will then add some kind of serialization or caching once it's working so I don't have to re-run the slow bits every time. I'll often also use nbdev to export a .py script from the notebook so it's easy to re-run the whole thing from scratch.
(BTW we also released something today that's particularly helpful for this workflow: https://fastai.github.io/execnb/ . Basically, it's a parameterised notebook runner. It doesn't rely on Jupyter or nbclient or nbconvert. It's in the same general category as Papermill, but it's much more lightweight and requires learning far fewer new concepts.)
Re: Nbdev: Create delightful software with Jupyter Notebooks
#9A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post: https://www.fast.ai/2022/07/28/nbdev-v2/ Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
When I looked into it a couple years ago, it was possible to configure nbdev to use e.g. GitLab instead of GitHub for those of us who can't use GitHub for whatever reason. Is this still possible with the rewrite? Any major things we'd be missing out on?
And thanks for putting together such an awesome resource, I'm excited to try kicking the tires on it again!
Re: Nbdev: Create delightful software with Jupyter Notebooks
#10A from-scratch rewrite of nbdev was released today -- details on what's new are in the launch post: https://www.fast.ai/2022/07/28/nbdev-v2/ Let me know if you have any questions or comments -- I'm the lead dev on the project. (If you're in the "I don't like notebooks" camp, please watch this first: https://www.youtube.com/watch?v=9Q6sLbz37gk )
When I looked into it a couple years ago, it was possible to configure nbdev to use e.g. GitLab instead of GitHub for those of us who can't use GitHub for whatever reason. Is this still possible with the rewrite? Any major things we'd be missing out on? And thanks for putting together such an awesome resource, I'm excited to try kicking the tires on it again!
Frankly it's not something I've been working on -- the GitLab support in v1 was added and maintained by the community. I certainly want it to work (even although I'm not a GitLab user myself), so if you try it and find it doesn't, please send us PRs/Issues if you can.