Git and Jupyter Notebooks Guide
reviewnb.com
Git and Jupyter Notebooks Guide
1–10 of 49 posts
Re: Git and Jupyter Notebooks Guide
#2Curious that they discuss several options, but ignore the totally obvious one: just use jupytext [0]. Jupytext is a (tiny) jupyter extension that reads/writes notebooks as python files, with text cells being represented as comments. With jupytext, you do away with the stupid .ipynb format. As long as you don't need to save the cell outputs, which is the case for version control, jupytext is the way to go.
People: pip install jupytext. All your python files will become notebooks, and your notebooks will become python files.
Re: Git and Jupyter Notebooks Guide
#3There is a draft JEP (Jupyter Enhancement Proposal) for Markdown-based notebooks (https://github.com/jupyter/enhancement-proposals/pull/103) which will make it a little more RMarkdown-like.
Re: Git and Jupyter Notebooks Guide
#4If you don't need to "commit" the output, you can just use nbconvert[0]:
jupyter nbconvert --clear-output --inplace my_notebook.ipynb
Re: Git and Jupyter Notebooks Guide
#5Curious that they discuss several options, but ignore the totally obvious one: just use jupytext [0]. Jupytext is a (tiny) jupyter extension that reads/writes notebooks as python files, with text cells being represented as comments. With jupytext, you do away with the stupid .ipynb format. As long as you don't need to save the cell outputs, which is the case for version control, jupytext is the way to go. People: pip…
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
Re: Git and Jupyter Notebooks Guide
#6Curious that they discuss several options, but ignore the totally obvious one: just use jupytext [0]. Jupytext is a (tiny) jupyter extension that reads/writes notebooks as python files, with text cells being represented as comments. With jupytext, you do away with the stupid .ipynb format. As long as you don't need to save the cell outputs, which is the case for version control, jupytext is the way to go. People: pip…
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
Why would you commit the outputs into git? That would be like committing compiled binary objects or pdfs. Of course the outputs are useful, but you just want to commit the sources.
The .ipynb stores inputs and outputs together in an unholy way. It is much cleaner to separate them. The inputs are python (or markdown) files that you can edit with a text editor and version control with git. The outputs are html, pdf, or whatever you want to nbconvert to and share.
The .ipynb file would only be useful if you want to share a stateful notebook, whose state cannot be easily reproduced by the people who you share it with. But that would be really bizarre and definitely in bad taste. Sharing the .ipynb is akin to sharing your .pyc files.
I love working with notebooks, but as a measure of hygiene I avoid .ipynb files altogether.
Re: Git and Jupyter Notebooks Guide
#7Curious that they discuss several options, but ignore the totally obvious one: just use jupytext [0]. Jupytext is a (tiny) jupyter extension that reads/writes notebooks as python files, with text cells being represented as comments. With jupytext, you do away with the stupid .ipynb format. As long as you don't need to save the cell outputs, which is the case for version control, jupytext is the way to go. People: pip…
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
I use jupytext paired with ipynb files. Only store the .py files in git. The ipynb files act as a local cache of outputs. Outputs are loaded from the ipynb even if you open the .py notebook.
Re: Git and Jupyter Notebooks Guide
#8Earlier quoted context omitted.
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
Why would you commit the outputs into git? That would be like committing compiled binary objects or pdfs. Of course the outputs are useful, but you just want to commit the sources. The .ipynb stores inputs and outputs together in an unholy way. It is much cleaner to separate them. The inputs are python (or markdown) files that you can edit with a text editor and version control with git. The outputs are html, pdf, or…
Most of the time I agree, but if you want to e.g present a tutorial as a webpage, having an ipynb with both inputs and outputs becomes a feature. You can even bundle into docs e.g. https://moj-analytical-services.github.io/splink/demos/02_Ex...
Re: Git and Jupyter Notebooks Guide
#9Earlier quoted context omitted.
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
Why would you commit the outputs into git? That would be like committing compiled binary objects or pdfs. Of course the outputs are useful, but you just want to commit the sources. The .ipynb stores inputs and outputs together in an unholy way. It is much cleaner to separate them. The inputs are python (or markdown) files that you can edit with a text editor and version control with git. The outputs are html, pdf, or…
Some people use the outputs like documentation, since github renders the notebook contents nicely in the browser. I agree it is not the best practice in many situations, I like using it on occasion though.
Another alternative if you want the outputs is to use nbconvert to convert the output to markdown, https://andrewpwheeler.com/2021/09/06/using-jupyter-notebook...
Re: Git and Jupyter Notebooks Guide
#10Earlier quoted context omitted.
What happens to the outputs in this case? I found the outputs to be both the most useful parts of notebooks, but also the most troublesome for diffing and versioning.
Why would you commit the outputs into git? That would be like committing compiled binary objects or pdfs. Of course the outputs are useful, but you just want to commit the sources. The .ipynb stores inputs and outputs together in an unholy way. It is much cleaner to separate them. The inputs are python (or markdown) files that you can edit with a text editor and version control with git. The outputs are html, pdf, or…
Sometimes I work on software development, and this mindset («the only valuable asset is the code») makes total sense. But if I work on analytics / datascience projects, the analysis including outputs could be time consuming to run, validate, and visualize. In these cases, it might be required to version the outputs.
I’ve never used jupyter for taking notes in a lab setting, but with more and more instruments being computer/network connected, I imagine this would make total sense - put your data and notes with your analytics work.
Many jupyter users are not «software developers», they just use code to perform their work.