Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jupyter, Mathematica, and the Future of the Research Paper

paulromer.net

Post reply on HN
Print view View original

81–90 of 179 posts

Re: Jupyter, Mathematica, and the Future of the Research Paper

#81

Post by dahart »

tincholio wrote: post #50
sanderjd wrote: post #47

Earlier quoted context omitted.

I used MathCAD in physics and chemistry classes in college. I frequently wonder what happened to it, because I share your perception that while Jupyter is very nifty, it is hamstrung by the limitations of the browser environment. I'm waiting impatiently for the coming revolution of non-web collaborative internet-connected rich client applications.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

> org-babel capabilities blows Jupyter out of the water ... Besides all this, it's just a plain text format

This feels like the argument against Slack in favor of IRC. I don't doubt it's true for you that org-babel is better, but there isn't really even that much comparison between Jupyter and org-babel, they are very different things.

One of the more compelling reasons to use Jupyter is inline images and plots. Another one of the more compelling reasons to use it is it basically comes bundled with everything you need, python with it's own virtual environment that doesn't modify your system python, and a bunch of amazing libraries, numpy, scipy, matplotlib. Not needing to know emacs helps too, but that's really low on the list.

> Jupyter uses a rather obtuse js on format which is not practical to work on directly.

This is a wierd thing to say. Jupyter has download as .py right in the file menu. Not to mention download as Markdown, HTML, LaTeX or PDF, if you want to present or share instead of export the python.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#82

Post by currymj »

bloaf wrote: post #21
ChrisRackauckas wrote: post #19

Earlier quoted context omitted.

It's not, but for most people it may not matter. Mathematica seems to have a much larger set of integrals, differential equations, special functions, etc. that it can recognize. So as much as I dislike the language itself, I do keep Mathematica installed because in many cases SymPy cannot handle the transformation. On the other hand, open source tools are catching up. Part of the SymPy project is SymEngine which is a…

Re: Speed Mathematica is undergoing a compiler re-vamp, which may change that.

one pretty clearly inspired by Julia's approach, from what I can understand.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#83

Post by FractalLP »

fdej wrote: post #60
gaius wrote: post #41

Earlier quoted context omitted.

The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly Have there been any high profile failures root-caused to Mathematica (or MATLAB or any similar product) getting its sums wrong? I can find any news stories etc. Plenty of serious calculations were and are done on “closed source” HP and TI calculators too…

Here is an article from a couple of years ago about how one group of mathematicians were misled by (and had to spend some time tracking down) a bug in Mathematica determinant evaluation: http://www.ams.org/notices/201410/rnoti-p1249.pdf

Thanks for posting. I'm sure bugs like this show up in any and all software. I do wish they had a public bug tracker though.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#84

Post by abhirag »

emj wrote: post #54
tincholio wrote: post #50

Earlier quoted context omitted.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

Requiring basic Emacs knowledge is just too much, just looking through the features of org-babel I can't really find anything that makes it better suited. Except not having to bother with a webbrowser, that is a nice plus, but hardly important for most people I work with.

I have used both Jupyter Notebooks and org mode with org-babel extensively and I agree with the OP regrading the fact that the org-babel workflow is vastly superior, OP did point out a few features which org mode workflow has and Jupyter Notebooks don't but I will try and provide a comprehensive list:

1. Plain text format, git and git diffs work

2. You can combine many languages in a single document, and every code block can be part of a separate session, as an analogy to Jupyter Notebooks, you can have multiple kernels backing a single notebook and you can decide what kernel you want the current code block to run in.

3. You can edit a code block in the major mode for that language, i.e. you get all the features of Emacs while editing code: documentation, auto-complete, snippets and anything Emacs can do, and Emacs can do a lot :)

4. You can have internal and external links to any part of the document (or any other org-mode file) within the editor which get exported as links in the HTML file too. Want to refer to a code block you used before, just name it and drop a link. Extremely useful in binding the whole document together.

5. Literate Programming support -- You can decide the order the concepts are introduced in according to the human reader, not according to the execution order the machine demands it to be in:

  #+NAME: named_code_block :eval no
  function_not_defined_yet()

  #NAME: complete_code_block
  def function_not_defined_yet():
      print("nice function innit?")
  
  >
The > gets expanded to whatever you defined it and you control the way you want to structure the document to be the most readable. You can keep working backed by a REPL in the initial stages and then extract(tangle in literate programming speak) to a file, again in the order you want using the > (NOWEB syntax). So one org-mode can generate your whole project if you wish so.

6. With the internal and external links and > (NOWEB syntax) the org-mode file is closer to being a hypertext file than Jupyter Notebook even though Jupyter Notebook is the one running in a browser.

I have covered only the major features of org-babel, I haven't even covered all the features. I love Jupyter Notebooks too, but org-babel is something else. I am currently working on a toy ray tracer in Clojure in literate programming style and loving every moment :)

Re: Jupyter, Mathematica, and the Future of the Research Paper

#85

Post by dahart »

carreau wrote: post #14
dahart wrote: post #8

> The tie-breaker is social, not technical. The tie-breaker is financial. Jupyter is winning because it's free, not because it's social. It becomes social because of widespread adoption, and it get's widespread adoption because it's free. I love Jupyter, love love love. But there's a lot of hyperbole and opinion here. Mathematica is just a for-profit business, it's that simple. And it wouldn't be fair to deny the exa…

> it's free Well for you maybe – and we strongly believe it should be – but it's built on top of thousand of volunteers hours, grants money (Thanks Sloan, Helmsley, Moore) and donation from companies (Anaconda, microsoft...) and individual, and partners. NumFOCUS ( https://www.numfocus.org/ ) manage all of that it's a 501c3 tax deductible ! If Jupyter is of help to you (or your company, organisation) think about cont…

Thank you for all your hard work!!

Understanding what it took for Jupyter to be where it is, and how much work it took, who pushed for it, and how many people had to work to make it happen, this is important.

You're absolutely right to emphasize that it's not free to build. I worry that promoting a mentality like the author's that open source and free to use is honest, and for-profit is "dishonest" dramatically undervalues and undermines your work as well as the funding model. If we treat Jupyter like it's our right, and not a miracle of funding and hard work, we will make it more difficult for Jupyter to attract help and to succeed.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#86

Post by tincholio »

dahart wrote: post #81
tincholio wrote: post #50

Earlier quoted context omitted.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

> org-babel capabilities blows Jupyter out of the water ... Besides all this, it's just a plain text format This feels like the argument against Slack in favor of IRC. I don't doubt it's true for you that org-babel is better, but there isn't really even that much comparison between Jupyter and org-babel, they are very different things. One of the more compelling reasons to use Jupyter is inline images and plots. Anot…

> One of the more compelling reasons to use Jupyter is inline images and plots

You can do those in org, since forever. I meant that the files themselves are plain text, which is good for sharing, version control, etc. ipynb files embed images with Base64 encoding, which makes seeing diffs a major PITA.

>This is a wierd thing to say. Jupyter has download as .py right in the file menu. Not to mention download as Markdown, HTML, LaTeX or PDF, if you want to present or share instead of export the python.

You can export org to those formats and more, but the main part is that you can organize your code (e.g, tangling to different files, etc.), and use other languages than python (or whatever other kernel you're running in Jupyter).

Plus, you get to use a proper text editor instead of editing on a browser textarea.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#87

Post by tincholio »

gaius wrote: post #73
tincholio wrote: post #50

Earlier quoted context omitted.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

combine several languages in a single document without any issues You can do this with RMarkdown as well, despite the name, and run notebooks in RStudio. Much better experience than Jupyter in a browser. https://yihui.name/knitr/demo/engines/

I haven't used Knitr (I do my R stuff from within Emacs, and org is a natural choice there), but it does look pretty powerful

Re: Jupyter, Mathematica, and the Future of the Research Paper

#88

Post by tincholio »

abhirag wrote: post #84
emj wrote: post #54

Earlier quoted context omitted.

Requiring basic Emacs knowledge is just too much, just looking through the features of org-babel I can't really find anything that makes it better suited. Except not having to bother with a webbrowser, that is a nice plus, but hardly important for most people I work with.

I have used both Jupyter Notebooks and org mode with org-babel extensively and I agree with the OP regrading the fact that the org-babel workflow is vastly superior, OP did point out a few features which org mode workflow has and Jupyter Notebooks don't but I will try and provide a comprehensive list: 1. Plain text format, git and git diffs work 2. You can combine many languages in a single document, and every code b…

You saved me a whole lot of typing! Thanks :)

I'd add that you can also benefit from other aspects of org, such as project management functionality, outline editing, table editing, tables with formulas, direct git integration via Magit, etc., etc.

I agree that getting people on Emacs is a non-minor issue, but using something like Spacemacs with CUA-mode enabled could go a long way towards acceptance.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#89

Post by jjgreen »

starpilot wrote: post #40
askvictor wrote: post #27

Earlier quoted context omitted.

While I totally hear you regarding the pain of python modules (particularly on Windows), the point of python 'distributions' like anaconda and canopy is to bring the kitchen sink along, kind of like mathematica. The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly. To be replicable, science involving data…

> The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly. To be replicable, science involving data needs to use open source tools. Excel and MATLAB can't be used for real science?

Matlab's fine, but Excel has (had?) some serious defects in the basic statistical functions, in particular, it coundn't reliably calculate a standard deviation.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#90

Post by tunaoftheland »

sanderjd wrote: post #47
gaius wrote: post #37

Earlier quoted context omitted.

The tie-breaker is financial. Jupyter is winning because it's free This is it really Back in the 90s I was using a program called MathCAD, it provided a “notebook” interface by running as a plugin to Word 6. In terms of general usability and experience, 20-years-ago blows away modern-day Jupyter and it’s silly “cells” interface, which it does not because it’s better but because it’s trying to force itself into a web…

I used MathCAD in physics and chemistry classes in college. I frequently wonder what happened to it, because I share your perception that while Jupyter is very nifty, it is hamstrung by the limitations of the browser environment. I'm waiting impatiently for the coming revolution of non-web collaborative internet-connected rich client applications.

As an alternative to Jupyter UI and Emacs, Hydrogen (https://github.com/nteract/hydrogen) could be viable. It runs as an Atom extension and connects to a Jupyter server instance. I haven't used it for anything other than a minimal project, but I preferred its UX to that of the browser interface of Jupyter. Atom isn't my favorite editor, but it's pleasant for this particular use case. Looks like the same team also offers an Electron-based application instead of an Atom extension.
Post reply on HN