Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Future of Notebooks: Lessons from JupyterCon

willcrichton.net

Post reply on HN
Print view View original

61–70 of 158 posts

Re: The Future of Notebooks: Lessons from JupyterCon

#61

Post by kgwgk »

gaius wrote: post #6
jakelarkin wrote: post #3

I understand why they became popular, but as a software engineer considering how they work, I am just full of disappointment. we're going to spend the next ten years re-inventing every single software engineering best practice for jupyter's weirdo environment.

In the mid-90s I was a Mech Eng undergrad using a program called MathCAD which provided a "notebook" interactive computation mixed with text environment by running as a Word plugin. In 2018 I use Jupyter and it's not clear where the progress has been. There are too many compromises trying to make it work in a web browser. For what Jupyter is for I find that RStudio or Spyder are infinitely superior. I do interactive…

Unfortunately RStudio also makes compromises to make it work in a web browser. A native tool could be even better.

Re: The Future of Notebooks: Lessons from JupyterCon

#62

Post by shawn »

seanmcdirmid wrote: post #39
stevesimmons wrote: post #38

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…

> And also more code editing to move from IDEs into the Jupyter Jupyter, and all REPLS in general, rely on a concrete top-level execution context to provide useful feeedback. Much of the code written outside of data science doesn’t really have that, which is why you don’t see notebooks being used for software development very often.

I'm not sure this is as limiting as it sounds. You can import whatever modules you want from within a notebook, and most projects tend to be modular. (Those that aren't can usually be shimmed pretty quickly.)

Re: The Future of Notebooks: Lessons from JupyterCon

#63

Post by michihuber »

To everyone frustrated with notebooks, I urge you to check out nextjournal.com:

- Notebooks are automatically versioned

- You can reference and reuse parts of other notebooks immutably through something they call "Transclusions": https://nextjournal.com/nextjournal/transclusions

- The technology-stacks underlying each notebook are immutable, meaning that notebooks work on any machine (no "hidden" dependencies).

- You can explicitly reference other results instead of relying on global state, making execution order irrelevant.

It's currently in private beta, but people are starting to use it in production. (EDIT: You can sign up using the code curryon2018)

(Disclaimer: friends of mine are building this and I used to work on it in the past.)

Re: The Future of Notebooks: Lessons from JupyterCon

#64

Post by fsloth »

ekianjo wrote: post #40
gaius wrote: post #20

Earlier quoted context omitted.

People do that in Excel too and the next thing you know a spreadsheet is managing a portfolio or being used as the basis for published science!

Its not the fault of the tools if users dont know any better.

So what would you suggest a financial modeler, who has no experience of any programming environments, to use instead?

The value of Excel is that is a zero-config tool, available everywhere as 'standard' business installation, allows very quick iteration with visual output in certain range of tasks, is battle tested in millions of computers... etc. And everyone else is using it too.

For a programmer it's easy to suggest to just pick some good language. Once you know two programming languages you can navigate pretty soon fluently in third.

But picking up the first language fluency? That's hard. You can't just suggest to use python or such. You need to provide a tool that holds hands, has lots tutorials available etc.

Visual programming is not necessary the answer either. I've understood that without discipline LabView programs become actual visual spaghetti pretty quickly.

Re: The Future of Notebooks: Lessons from JupyterCon

#65

Post by alper111 »

aluren wrote: post #60

It would seem that notebooks were specifically tailored for my use cases (being in academia and doing data analysis), in practice I found that I drift away from notebooks every time I try them and stick to a good old console (ipython or bash) instead. Here are a few things that irked me: -It's browser based. Yuck. This means you can wait upwards of ten seconds for a session to load up every time you want to start one…

I agree that probably working on bash feels more snappy but Jupyter is not that clunky as you mentioned. My laptop is a mediocre one and I can run the Jupyter, navigate to the code and open the editor in total like 3 seconds.

In my experience, kernels work just fine unless you are doing something you shouldn't. My kernel problems occur at points where I have a memory issues. Though I agree with you that you don't understand what's wrong once the kernel dies.

Re: The Future of Notebooks: Lessons from JupyterCon

#66

Post by Ixio »

stevesimmons wrote: post #38

I don't get the people here saying don't use Jupyter notebooks, or they are bad software engineering. So much Python development is trying snippets of code in a REPL as you introspect live objects, then once they're right pasting them into the IDE. All my Jupyter notebooks are like that, where my code starts as cells of a line or two, as I check each output. Then I coalesce them into a function (which avoid the probl…

> The Jupyter notebooks are version controlled like my .py modules.

That's interesting, could you give us some more details ? Last time I tried to put a Jupyter notebook under git it was as mess. Do you use another tool than git or have they made tools to help with version control ? Or is it just your workflow that helps, like emptying all cell results before saving ?

Re: The Future of Notebooks: Lessons from JupyterCon

#67

Post by LittlePeter »

h4b4n3r0 wrote: post #55

Even in its present state, Jupyter is strictly better than plain REPL, especially if you need visualization. It's like Python in general: it was not designed to write 10-100KLOC programs. Python was designed for scripting. Jupyter was designed for interactive data exploration, and to create a record of results which you can view without re-executing the cells. Is it error prone? Yes. But then so is REPL.

> Python was designed for scripting.

Can you provide a citation?

Re: The Future of Notebooks: Lessons from JupyterCon

#69

Post by aluren »

alper111 wrote: post #65
aluren wrote: post #60

It would seem that notebooks were specifically tailored for my use cases (being in academia and doing data analysis), in practice I found that I drift away from notebooks every time I try them and stick to a good old console (ipython or bash) instead. Here are a few things that irked me: -It's browser based. Yuck. This means you can wait upwards of ten seconds for a session to load up every time you want to start one…

I agree that probably working on bash feels more snappy but Jupyter is not that clunky as you mentioned. My laptop is a mediocre one and I can run the Jupyter, navigate to the code and open the editor in total like 3 seconds. In my experience, kernels work just fine unless you are doing something you shouldn't. My kernel problems occur at points where I have a memory issues. Though I agree with you that you don't und…

>In my experience, kernels work just fine unless you are doing something you shouldn't.

Here's the thing. I'm sure many of the complaints I voiced can be chalked up to lack of experience. But even if it weren't for the issues about documentation and troubleshooting, I'd still have yet another tool to manage, learn to use, learn best practices, etc. Instead of just getting things done with my tool it just gets in the way and tells me there are things I should and shouldn't do. It takes time to learn about it.

And now, I'm not sure the investment is worth the effort. Despite its warts, everyone uses bash and it's really easy to learn about it and translate that knowledge into many applications. And, it does the job when I ask it to. Same for Python, IDE/editor usage, LaTeX, and all the other toys in your typical academia toolbox. I'm all for learning. But time is short, deadlines loom close, and the good enough is often the better's worst enemy. And when it turns out the better isn't actually better, welp.

Re: The Future of Notebooks: Lessons from JupyterCon

#70

Post by h4b4n3r0 »

LittlePeter wrote: post #67
h4b4n3r0 wrote: post #55

Even in its present state, Jupyter is strictly better than plain REPL, especially if you need visualization. It's like Python in general: it was not designed to write 10-100KLOC programs. Python was designed for scripting. Jupyter was designed for interactive data exploration, and to create a record of results which you can view without re-executing the cells. Is it error prone? Yes. But then so is REPL.

> Python was designed for scripting. Can you provide a citation?

I don’t provide citations for such obvious things. If you’re interested, you can find the origin story as written by Guido himself in Python FAQ using any of the available search engines.
Post reply on HN