Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimating Number of Jupyter Notebooks on Github

kyso.io

Post reply on HN
Print view View original

91–100 of 135 posts

Re: Estimating Number of Jupyter Notebooks on Github

#91

Post by jerednel »

KyleOS wrote: post #85
hodder wrote: post #64

Very cool work here. This is a pretty epic post, so please do not take this the wrong way. I was under the impression that FB Prophet was optimal for significantly seasonal time series data. Honestly given the fickle nature of these kind of growth patterns beyond the very near term, an ARIMA with a flat vol or a simple eyeball extrapolation in my experience as a quant would likely generate just as reasonable/reliable…

Hey, I posted the notebook by the OP. Thank you for your feedback! You're correct in saying that FB Prophet is for forecasting time series with strong seasonal effects. FB Prophet was the model used in the original script I found & the main point here was simply to make the notebook more readable on kyso, which has quite a few non-technical readers. I've worked a lot with ARIMAs before for financial/economic data and…

Did you try running the forecast with a log ceiling to control for the trajectory a bit? Or would that only be a concern of yours if you had to forecast past a couple of years? I find that when I use Prophet to forecast down to the day I end up creating initial forecasts with heavy log ceilings to prevent unreasonable estimates of the future and then end up removing the ceiling once enough history is established to provide a resaonable baseline.

Re: Estimating Number of Jupyter Notebooks on Github

#92

Post by eoinmurray92 »

mike_ivanov wrote: post #86
eoinmurray92 wrote: post #55

Earlier quoted context omitted.

Mostly so you don't need to convert to PDF and so that you can host the reports in a central place where everyone can read them technical or not. Like an internal wiki for a companies data-science where the technical people can communicate their work to the non-technical people with a pretty seamless experience

This is not the experience non-technical senior leadership people are looking for, unless you are a 10-people startup.

Im not sure - we have large teams using Kyso as a knowledge base for data-science work and there's also Airbnb's knowledge-repo which originally inspired us so from my point of view there is decent evidence for the need for this

Re: Estimating Number of Jupyter Notebooks on Github

#95

Post by blueboo »

dec0dedab0de wrote: post #90
julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

I disagree. While your points are valid for presentations, I believe that notebooks should first and foremost be used for exploratory computing. Notebooks are my goto tool for whenever I need to do something with a computer and any of the following apply: * I'm not quite sure what or how. * It will likely be a one-off * I need it now . * Someone is watching me, to learn how I do it. I would go a step further and say…

Indeed, when the alternative is the interpreter, the structure notebooks offer and the (loose) process they encourage makes for a vastly better solution for recording and capitalizing on your work.

Re: Estimating Number of Jupyter Notebooks on Github

#96

Post by localhost »

djhworld wrote: post #76
localhost wrote: post #35

Earlier quoted context omitted.

What specifically do you hate about Jupyter? Is it out of order execution exacerbating the "hidden state" problem? If so, and if you already use VS Code, I encourage you to try out our Python VS Code extension. We have an "Interactive Python Window" mode https://code.visualstudio.com/docs/python/jupyter-support that we showed a lot of folks at pycon last week and even among the "I don't like Jupyter" crowd, it was qu…

This is really cool. I tend to do a lot of exploratory work in Jupyter stuff, but find the whole process really annoying and cumbersome to set up - just been playing around with this VS code extension and it seems really neat! I've been using nteract a lot recently but I'm gonna switch to VS Code now, at least that reduces one Electron app eating up my memory

Yay for less electrons! We're also using nteract for a future release of Azure Notebooks, and we're already using nteract in the VS Code extension. BTW, by nteract I mean nteract as a suite of components for building Jupyter things vs. nteract the electron app. We recently hired the awesome Safia Abdalla to help us contribute back to the nteract community and things have been going great!

Re: Estimating Number of Jupyter Notebooks on Github

#97

Post by analog31 »

julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

While I don't follow #2, I suggest two more item:

Take the time to learn some good programming practices. If you're lucky and fall in love with scientific programming, your programs will grow to a point where they are unmanageable without some techniques borrowed from computer science and software engineering.

Have someone show you the "out of order execution" and "hidden state" failures.

Re: Estimating Number of Jupyter Notebooks on Github

#98

Post by rodonn »

KyleOS wrote: post #19
turingbike wrote: post #14

Earlier quoted context omitted.

Aren't Jupyter notebooks R notebooks? Jupyter stands for "Julia, Python, R" I believe

I think the file extension scraped on Github is only .ipynb, which is only python notebooks right?

I think most people save under this extension even if they are using a different kernel (i.e. they are running R, Julia, Matlab, etc. code in the notebook).

Re: Estimating Number of Jupyter Notebooks on Github

#99

Post by nerdponx »

kbd wrote: post #30

If you ever put notebooks in source control, you owe it to yourself to try the text-based notebooks supported in Visual Studio Code[1]. They're round-trippable with real (i.e. browser-based) notebooks, yet are much better for collaboration, diffing, and editing. [1] https://code.visualstudio.com/docs/python/jupyter-support

The R ecosystem also has a text-based notebook format that they call R Markdown, with a long lineage going back to the TeX days.

It never caught on outside of the R community, but the format itself is language-agnostic.

https://rmarkdown.rstudio.com/

Re: Estimating Number of Jupyter Notebooks on Github

#100

Post by _raoulcousins »

julienchastang wrote: post #61

In the same spirit as “Effective Java” and “Effective C++” we need to have a book entitled “Effective Jupyter Notebooks”. Here are some of my items below. Maybe this sub-thread can come up with an outline for this book. Item #1 Writing a notebook is foremost an exercise in expository writing. Make sure the writing is high quality is the first objective when writing a notebook. This is the Knuth’s literate programming…

Obligatory Joel Grus: slides from his 'I don't like notebooks' talk. He has good follow-up talks too.

https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...

Post reply on HN