Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimating Number of Jupyter Notebooks on Github

kyso.io

Post reply on HN
Print view View original

121–130 of 135 posts

Re: Estimating Number of Jupyter Notebooks on Github

#121

Post by mike_ivanov »

eoinmurray92 wrote: post #92
mike_ivanov wrote: post #86

Earlier quoted context omitted.

This is not the experience non-technical senior leadership people are looking for, unless you are a 10-people startup.

Im not sure - we have large teams using Kyso as a knowledge base for data-science work and there's also Airbnb's knowledge-repo which originally inspired us so from my point of view there is decent evidence for the need for this

Ah, that makes sense. So, basically you replaced a dashboard effort with a whole bunch of readonly notebooks, thus distributing the information delivery job among the peers outside of your DS team. Clever.

Re: Estimating Number of Jupyter Notebooks on Github

#122

Post by _coveredInBees »

localhost wrote: post #35
funkythingsss wrote: post #5

I hate jupyter notebooks. Joel Grus puts it perfectly: https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh... past hn discussion: https://news.ycombinator.com/item?id=17856700

What specifically do you hate about Jupyter? Is it out of order execution exacerbating the "hidden state" problem? If so, and if you already use VS Code, I encourage you to try out our Python VS Code extension. We have an "Interactive Python Window" mode https://code.visualstudio.com/docs/python/jupyter-support that we showed a lot of folks at pycon last week and even among the "I don't like Jupyter" crowd, it was qu…

Nice work on this! I think apart from the very valid points raised by Joel Gru in his presentation slides, Jupyter Notebooks are woefully inadequate for exploring and understanding code that utilizes libraries. I'll take the FastAI library as an example since Jeremy Howard is the one who took Joel to task regarding criticizing Notebooks:

If I view any fastai notebooks taken from their repo, there are a million imports (having a bunch of `import *` statements in the fastai lib doesn't help things) and methods and classes keep popping up out of nowhere. Good luck making sense out of them in a notebook. At least in Pycharm or VS Code, it's one Ctrl+Click away from viewing the relevant source code and having a somewhat better idea of what is going on.

Debugging is woefully inadequate compared to the Pycharm experience (VS Code is slowly catching up with Pycharm on that front).

I've only found 2 good uses for Jupyter Notebooks:

1. As a scratch-pad to try out things without any plans to utilize directly or share the code with anyone

2. As a means to write well documented examples/code with a bunch of markdown...especially when I'm modeling things that have a lot of equations, where the LATEX support is a boon and makes the documentation in the notebook far superior to what you could achieve in a regular .py script.

In almost every other instance, you are better off using a full-blown IDE with a far superior development environment, ability to have venvs, superior debugging, superior code management/refactoring and most importantly, much better reproducibility.

All that being said, I think the direction in VSC is definitely a great step in the right direction. I honestly love VSC but can't give up PyCharm yet as it is still a long way ahead of VSC when it comes to Python features (much better linting, much better management of venvs and run configs, more powerful debugging experience... though VSC is getting pretty good, much better refactoring support and PEP-8 reformatting/linting). I really hope there is more of a push within MS to continue improving the Python experience in VS Code. Nothing would make me happier than to consolidate my work in VSCode! Keep up the awesome work!

Re: Estimating Number of Jupyter Notebooks on Github

#123

Post by xvilka »

claytonjy wrote: post #34
kbd wrote: post #30

If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round-trippable with real (i.e. browser-based) notebooks, yet are much better for collaboration, diffing, and editing. [1] https://code.visualstudio.com/docs/python/jupyter-support

How does this compare to Jupytext? I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?

Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.

Re: Estimating Number of Jupyter Notebooks on Github

#124

Post by xvilka »

syntaxing wrote: post #32

On the topic of Jupyter Notebooks, Is there something similar to a paid version of Google's CoLab? CoLab is so awesome for creating prototypes and even better since it's free. However, there is no paid alternative that I have seen. I do not want to have to deal with setting up my own VM or server. The way that CoLab is perfect for what I need.

JupyterLab: https://github.com/jupyterlab/jupyterlab

Re: Estimating Number of Jupyter Notebooks on Github

#125

Post by linalgmixer »

xvilka wrote: post #123
claytonjy wrote: post #34

Earlier quoted context omitted.

How does this compare to Jupytext? I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?

Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.

But the purpose of Anaconda is to have a easily installable set of frequently used tool for a variety of data science tasks. It's not meant to be a minimalist package.

Re: Estimating Number of Jupyter Notebooks on Github

#126

Post by localhost »

_coveredInBees wrote: post #122
localhost wrote: post #35

Earlier quoted context omitted.

What specifically do you hate about Jupyter? Is it out of order execution exacerbating the "hidden state" problem? If so, and if you already use VS Code, I encourage you to try out our Python VS Code extension. We have an "Interactive Python Window" mode https://code.visualstudio.com/docs/python/jupyter-support that we showed a lot of folks at pycon last week and even among the "I don't like Jupyter" crowd, it was qu…

Nice work on this! I think apart from the very valid points raised by Joel Gru in his presentation slides, Jupyter Notebooks are woefully inadequate for exploring and understanding code that utilizes libraries. I'll take the FastAI library as an example since Jeremy Howard is the one who took Joel to task regarding criticizing Notebooks: If I view any fastai notebooks taken from their repo, there are a million import…

Thanks for the kind words!

I'm currently thinking about a model of notebooks / interactive programming where the default assumption is that you're using it as a scratchpad, i.e., you won't need to explicitly name the file in order to get the benefits of auto-save, but yet the file won't pollute your filesystem / project namespace until you choose to "keep" it. Hopefully this helps reduce the friction in the exploratory programming realm; my goal is to eliminate the "ConsoleApplicationX" directories (I'm a VS guy from way back so ...) and I think it helps with your scenario 1) above.

The Python VS Code extension team is well aware of the gaps that you list as well, and are working hard to narrow them with each release. We are definitely serious about improving the Python experience in VS Code. How times have changed :)

Thanks for the support and encouragement. And as always, if you find issues that aren't already in our github, feel free to add some more and keep the feedback coming!

Re: Estimating Number of Jupyter Notebooks on Github

#127

Post by mlevental »

kbd wrote: post #30

If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round-trippable with real (i.e. browser-based) notebooks, yet are much better for collaboration, diffing, and editing. [1] https://code.visualstudio.com/docs/python/jupyter-support

pycharm supports the same thing, with debug, but frustratingly doesn't allow running all cells

Re: Estimating Number of Jupyter Notebooks on Github

#128

Post by [deleted] »

julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

[deleted]

Re: Estimating Number of Jupyter Notebooks on Github

#129

Post by KyleOS »

aw3c2 wrote: post #111
eoinmurray92 wrote: post #10

Earlier quoted context omitted.

This is what we made Kyso for - the linked post is actually a Jupyter notebook itself, the code is hidden by default to make it readable to non-technical people but you can click on the "code hidden" button on the top right to see the code in full. If github is not working for you well for your notebooks. You can try Kyso by signing up and importing your notebook from Github directly on this page: https://kyso.io/git…

You made Kyso for making GitHub-hosted Jupyter Notebooks render when accessed with a web browser on github.com?

Well, it's one of the ways to post to Kyso, but yeah, you can synchronize your Github repositories to Kyso and choose the notebooks you want to have rendered. When you push changes to the repo, the sister post on Kyso will be automatically updated.

Re: Estimating Number of Jupyter Notebooks on Github

#130

Post by kalefranz »

xvilka wrote: post #123
claytonjy wrote: post #34

Earlier quoted context omitted.

How does this compare to Jupytext? I prefer pipenv to Conda, and I don't like having Jupyter(Lab) installed in each venv separately, so instead I only add `Ipykernel` to each venv and then use my system-level JupyterLab to access per-project kernels; seems like that wouldn't work here?

Exactly. Conda is the worst of Python world - installing gigabytes of unnecessary garbage every time.

Conda can create environments with literally nothing in them. If you’re putting the full-blown anaconda metapackage in every environment, then it sounds like you’re using conda wrong for your use case. Just tell conda to install the packages you actually need.
Post reply on HN