Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimating Number of Jupyter Notebooks on Github

kyso.io

Post reply on HN
Print view View original

111–120 of 135 posts

Re: Estimating Number of Jupyter Notebooks on Github

#111

Post by aw3c2 »

eoinmurray92 wrote: post #10
manaskarekar wrote: post #2

Off topic: Possibly something in my config, but I've recently got a lot of "Sorry, something went wrong. Reload?" when trying to view Jupyter notebooks on github itself. Seems to be working right now. I have used this as an alternate: https://nbviewer.jupyter.org/

This is what we made Kyso for - the linked post is actually a Jupyter notebook itself, the code is hidden by default to make it readable to non-technical people but you can click on the "code hidden" button on the top right to see the code in full. If github is not working for you well for your notebooks. You can try Kyso by signing up and importing your notebook from Github directly on this page: https://kyso.io/git…

You made Kyso for making GitHub-hosted Jupyter Notebooks render when accessed with a web browser on github.com?

Re: Estimating Number of Jupyter Notebooks on Github

#112

Post by gnulinux »

tincholio wrote: post #48

If only more people would use org-babel... If you're on emacs and like Jupyter, there's https://github.com/dzop/emacs-jupyter , which is pretty nice. I've been using it for a few days with Julia, and it works really nice. It also allows you to use different kernels from the same org-mode file, though I haven't tried to pass data between them yet (should be possible, though, at least it works in plain org-mode).

Emacs jupyter notebook mode (ein-mode) is pretty excellent too. That's my go-to way to connect to jupyter.

Re: Estimating Number of Jupyter Notebooks on Github

#114

Post by lmeyerov »

eoinmurray92 wrote: post #25
lmeyerov wrote: post #12

We use notebooks heavily for onboarding devs & data scientists to Graphistry, and I only see that increasing. Interestingly, for initial use, we increasingly start teams on their existing internal NB servers, and for new ones, they either start on Jupyter included in their Graphistry AMI or use Google Colab. So, very little outside of our quick start notebook skeletons hits GitHub. So... How many notebooks are actual…

How do you share and collaborate on the notebooks internally - I'd love to get your thoughts on our Kyso for teams system [1] if you would be willing to chat? [1] https://kyso.io/for-teams

Google Colab has solved the 90% for us. I wish it'd have context sharing across users and better default folder management, and probably other things, but free + usable + sharing has been amazing in + across partners.

Feel free to ping in a ~couple weeks, happy to chat.

Re: Estimating Number of Jupyter Notebooks on Github

#115

Post by hulahoof »

jsilence wrote: post #17
turingbike wrote: post #14

Earlier quoted context omitted.

Aren't Jupyter notebooks R notebooks? Jupyter stands for "Julia, Python, R" I believe

But the user is choosing a Jupyter Kernel upon creation and the code is run on either one of the available Kernels. No code mixing within one notebook at the current point.

I thought you could with apache toree? Maybe I misunderstand the project

Re: Estimating Number of Jupyter Notebooks on Github

#116

Post by bluejay2387 »

dec0dedab0de wrote: post #90
julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

I disagree. While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing. Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply: * I'm not quite sure what or how. * It will likely be a one-off * I need it now . * Someone is watching me, to learn how I do it. I would go a step further and say…

I agree with with your disagreement. I've always approached notebooks as a prototyping tool first and a presentation tool second. I see Jupyter more as an example of an early Interactive programming tool that just happens to be useful for presentation and teaching purposes. (https://en.wikipedia.org/wiki/Interactive_programming)

And regarding sharing -- particularly with notebooks you should be really cautious about sharing and opening them. The security model is not exactly bullet proof.

Re: Estimating Number of Jupyter Notebooks on Github

#117

Post by TeMPOraL »

dec0dedab0de wrote: post #90
julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

I disagree. While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing. Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply: * I'm not quite sure what or how. * It will likely be a one-off * I need it now . * Someone is watching me, to learn how I do it. I would go a step further and say…

Agreed. I've been increasingly doing that at work with org-mode. Sometimes I'll just open up a source block under a TODO item, other times I'll create a new heading or a new file for some stream of thoughts mixed with code blocks in various languages. It feels better than doing it in a pure source code file, with my thoughts noted as comments (though I do plenty of that too).

Re: Estimating Number of Jupyter Notebooks on Github

#118

Post by TeMPOraL »

bluejay2387 wrote: post #116
dec0dedab0de wrote: post #90

Earlier quoted context omitted.

I disagree. While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing. Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply: * I'm not quite sure what or how. * It will likely be a one-off * I need it now . * Someone is watching me, to learn how I do it. I would go a step further and say…

I agree with with your disagreement. I've always approached notebooks as a prototyping tool first and a presentation tool second. I see Jupyter more as an example of an early Interactive programming tool that just happens to be useful for presentation and teaching purposes. ( https://en.wikipedia.org/wiki/Interactive_programming ) And regarding sharing -- particularly with notebooks you should be really cautious abou…

> I see Jupyter more as an example of an early Interactive programming tool

Nitpick, but I hope you meant "young", not "early". Interactive programming is as old as (and today still primarily featured in) Lisp - i.e. twice as old as most of us here on HN.

Re: Estimating Number of Jupyter Notebooks on Github

#119

Post by hodder »

KyleOS wrote: post #85
hodder wrote: post #64

Very cool work here. This is a pretty epic post, so please do not take this the wrong way. I was under the impression that FB Prophet was optimal for significantly seasonal time series data. Honestly given the fickle nature of these kind of growth patterns beyond the very near term, an ARIMA with a flat vol or a simple eyeball extrapolation in my experience as a quant would likely generate just as reasonable/reliable…

Hey, I posted the notebook by the OP. Thank you for your feedback! You're correct in saying that FB Prophet is for forecasting time series with strong seasonal effects. FB Prophet was the model used in the original script I found & the main point here was simply to make the notebook more readable on kyso, which has quite a few non-technical readers. I've worked a lot with ARIMAs before for financial/economic data and…

Ah okay. Great work with plotly on visualization then. This is a great skill to have and something I need to work on. I’ve added Kyso to my favorite list. I look forward to your future posts.

I will likely publish something on my project using stale data given I work in a trading environment. The theory should be the same though. One of these days I’d like to actually write a solid white paper level research study and get published! One can dream!

Re: Estimating Number of Jupyter Notebooks on Github

#120

Post by martinzugnoni »

zeptomu wrote: post #78
martinzugnoni wrote: post #74

Earlier quoted context omitted.

We might add bigger paid tiers later if we decide to support business usage of it. For now it's only educational and we can deal with the costs, even without the credits. Containers used are small and get shut down on inactivity. So, we only need to care about concurrent users. Hope it makes sense.

> Containers used are small and get shut down on inactivity. How do you define inactivity? If I do $ nohup ./computational_intense_and_runs_for_100_hours.py & Do you just kill the process (or stop the container)? In essence Jupyter is a graphical rich shell, so you providing free *nix machines - don't underestimate how this feature can be exploited (e.g. CoCalc limits at least internet access for free instances).

First, that will use 100% of the CPU quota assigned to your user, which is really small.

Second, yes. The container will be killed after 10min unless we keep detecting activity of your user in the platform. So, basically the rule is: If we don't detect user's activity after 10mins we kill all containers for that user. You could hack this by doing periodical requests to the API to simulate activity, but at some point your JWT will be expired and requests will start failing.

In any case, other students won't be affected at all by the appropriate usage and we will end up banning your account at some point when we detect it.

We also limit the amount of parallel running containers to avoid unlimited containers running at the same time.

Do you see any drawbacks on this implementation? Happy to hear about possible improvements.

Post reply on HN