Jupyter, Mathematica, and the Future of the Research Paper • Hacker News Board

Re: Jupyter, Mathematica, and the Future of the Research Paper

#91

Post by ChrisRackauckas » Sun, Apr 15, 2018, 2:47 PM UTC

jhbadger wrote: post #56

ChrisRackauckas wrote: post #17

>Which reminds me. If you are a Julia enthusiast, how do you suppose the investors in this new language plan to make their big score? This is a weird jab at Julia. Open source software is woefully underfunded. Julia Computing was founded in the wake of Heartbleed where people learned that open source needs some kind of funding to keep developers alive (example article: https://arstechnica.com/information-technology/2…

Also the idea of companies making money off of open source isn't new or nefarious and often helps people who aren't even their customer. Red Hat is the classic example, and many organizations use CentOS, a distribution based off of Red Hat's distro without paying Red Hat anything. And in scientific computing there's RStudio, which makes a great open source IDE for R besides offering prducts and services for sale.

Wow, I asked for the data that led him to his conclusion about "the monopoly" and he blocked me. So much for open science...

https://twitter.com/paulmromer/status/985529525491654657

Re: Jupyter, Mathematica, and the Future of the Research Paper

#92

Post by todd8 » Sun, Apr 15, 2018, 2:49 PM UTC

tincholio wrote: post #50

sanderjd wrote: post #47

Earlier quoted context omitted.

I used MathCAD in physics and chemistry classes in college. I frequently wonder what happened to it, because I share your perception that while Jupyter is very nifty, it is hamstrung by the limitations of the browser environment. I'm waiting impatiently for the coming revolution of non-web collaborative internet-connected rich client applications.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

I share your opinion about the superiority of org-mode as a means to assemble a document out of various pieces. It's not hard to show nicely formatted code, the code's output, the math behind the code and perhaps a graph of the results all in a LaTeX/PDF document and or HTML. All of this can be done with a flat text file that is easy to save as a part of a project under source control.

What I wonder is why are Jupyter notebooks so popular? I've tried them, but perhaps I don't understand the workflow in the way people use them. Are they intended to be works in progress where one explores ideas and looks at results? This is how I've usually used them and Mathematica's notebooks as well.

For me though, my Jupyter or Mathematica notebooks end up with lots of false steps and dead-ends. To prepare a presentable form of the results I have to go back and edit the notebook to such a degree that it doesn't feel like working in a notebook as much as it is working in a complex document with a really weak set of editing tools. To be fair, I haven't used Jupyter or Mathematica for a couple of years and things may have changed. Maybe HN readers can help be understand how to more effectively use these tools.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#93

Post by abhirag » Sun, Apr 15, 2018, 2:57 PM UTC

tincholio wrote: post #88

abhirag wrote: post #84

Earlier quoted context omitted.

I have used both Jupyter Notebooks and org mode with org-babel extensively and I agree with the OP regrading the fact that the org-babel workflow is vastly superior, OP did point out a few features which org mode workflow has and Jupyter Notebooks don't but I will try and provide a comprehensive list: 1. Plain text format, git and git diffs work 2. You can combine many languages in a single document, and every code b…

You saved me a whole lot of typing! Thanks :) I'd add that you can also benefit from other aspects of org, such as project management functionality, outline editing, table editing, tables with formulas, direct git integration via Magit, etc., etc. I agree that getting people on Emacs is a non-minor issue, but using something like Spacemacs with CUA-mode enabled could go a long way towards acceptance.

I would love discussing the literate ray tracer I have been working on and the literate programming workflow I have made for it. Couldn't find your email in the profile, mine is abhirag@outlook.com, would be great to discuss if I can improve my workflow further :)

Re: Jupyter, Mathematica, and the Future of the Research Paper

#94

Post by dr_zoidberg » Sun, Apr 15, 2018, 2:58 PM UTC

FractalLP wrote: post #23

Long time user of Python here and recent user of Mathematica. Some observations I have are that they're both great. Python is a nice open source scripting language, but getting libraries to work can sometimes be a pain. Mathematica is basically install this and everything is included. The Mathematica documentation is amazing and it is really simple how to do most things. The whole iPhone "there is an app for that" is…

The best package for graph analysis in Python is NetworkX: http://networkx.github.io/

Re: Jupyter, Mathematica, and the Future of the Research Paper

#95

Post by tincholio » Sun, Apr 15, 2018, 3:06 PM UTC

todd8 wrote: post #92

tincholio wrote: post #50

Earlier quoted context omitted.

As a die-hard Emacs user, I think that org-mode with it's org-babel capabilities blows Jupyter out of the water, and it produces much better, readable output. You can use it with pretty much any language, and combine several languages in a single document without any issues. Besides all this, it's just a plain text format, and you can extract all the code into proper source files for later offline use, too ('tangling…

I share your opinion about the superiority of org-mode as a means to assemble a document out of various pieces. It's not hard to show nicely formatted code, the code's output, the math behind the code and perhaps a graph of the results all in a LaTeX/PDF document and or HTML. All of this can be done with a flat text file that is easy to save as a part of a project under source control. What I wonder is why are Jupyte…

> What I wonder is why are Jupyter notebooks so popular?

I often wonder the same. I'm now kinda forced to use them, since they're the tool of choice in my team, but it's a major pain. I think that they're "easy" (in the way Rich Hickey describes "easy" vs "simple") to get started with (just as python, btw), but their lack of flexibility comes back to bite you sooner or later.

For me, I've resorted to using a mix of EIN and ob-ipython, and then copy/pasting as needed to keep the stuff in synch with Jupyter. I'm now considering writing a simple ox-jupyter exporter, and maybe some import functionality, to avoid dealing with Jupyter directly.

John Kitchin has written a basic exporter (http://kitchingroup.cheme.cmu.edu/blog/2017/01/21/Exporting-...), but it doesn't support inline images (it should be simple to fix this), and it doesn't really adhere to the ox- style of exports, so I think I'll probably do my own instead of adapting his (it'll be a good learning experience, for sure).

Re: Jupyter, Mathematica, and the Future of the Research Paper

#96

Post by dahart » Sun, Apr 15, 2018, 3:07 PM UTC

tincholio wrote: post #86

dahart wrote: post #81

Earlier quoted context omitted.

> org-babel capabilities blows Jupyter out of the water ... Besides all this, it's just a plain text format This feels like the argument against Slack in favor of IRC. I don't doubt it's true for you that org-babel is better, but there isn't really even that much comparison between Jupyter and org-babel, they are very different things. One of the more compelling reasons to use Jupyter is inline images and plots. Anot…

> One of the more compelling reasons to use Jupyter is inline images and plots You can do those in org, since forever. I meant that the files themselves are plain text, which is good for sharing, version control, etc. ipynb files embed images with Base64 encoding, which makes seeing diffs a major PITA. >This is a wierd thing to say. Jupyter has download as .py right in the file menu. Not to mention download as Markdo…

I Agree with all those points. I've used org mode for a few things, but not for python notebooks. I remember seeing a great video a few years back on using python in org mode to make a research notebook. It really made me want to try org mode for literate / notebook style programming. I think it was this one: https://youtu.be/1-dUkyn_fZA

It sounds like org mode does not store images inline, since you don't like the b64 images in Jupyter? I imagine that could go both ways depending on what you want. Even for diffing, you might like to see that the image changed (though I generally agree that diffing images is not ideal). If you have to package images separately when sharing, then wouldn't inline images score a point for easier sharing?

> use other languages that python

Agreed, and that is powerful, but this also goes to my point. Jupyter is made for python and org mode isn't. Jupyter comes with python and libraries, where with Emacs you're on your own to figure out how to install and use different languages, and setup your environment so Emacs can see them. You're on your own to bring in images and plots, where Jupyter already understands commands that output images.

> Plus, you get to use a proper text editor instead of editing on a browser textarea.

Yes, but the editor is Emacs. :) For a lot of people the browser textarea is a bonus because it's the same thing you use everywhere. It's uncomplicated and non-technical and utterly consistent, even if lacking any power.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#97

Post by EGreg » Sun, Apr 15, 2018, 3:10 PM UTC

It always goes like this.

The initial solutions may be proprietary, and financed by investors. They have a business model so of course they don’t give everything away for free.

With time, enough people get together to build an open source alternative. And then like a snowball it eclipses everything proprietary that went before it.

What would a world look like that didn’t apply Capitalism to ideas?

One where companies couldn’t sue one another for Intellectual Property infringements. Like Waymo suing Uber.

One where self driving cars can incorporate improvements made by any other self driving car instead of putting people at risk reinventing the wheel.

Where the long tail of drug research leads to something.

Why would people release their findings? Because if they don’t, others will. And then they won’t get that small measure of input and control and attach their name to it. Jonas Salk is an exception in the biomedical field. Albert Einstein is the norm in physics.

The alternatives to Capitalism do not have to be Socialism. They can be SCIENCE. OPEN SOURCE. WIKI.

Collaboration instead of Competition.

I would like to see the same in web browser engines. WebKit instead of IE. And so on. When that happens, we all win.

Yes free software is good. Software, like knowledge, does not have to be scarce.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#98

Post by madengr » Sun, Apr 15, 2018, 3:13 PM UTC

gaius wrote: post #78

madengr wrote: post #77

Earlier quoted context omitted.

MathCAD went to shit in about 2000. I had been using it for 10 years then (started in DOS). They made some annoying UI changes and really increased the cost. The kicker was when they called me and accused me of using a fake license, when I had dutifully been paying support for both a home and work license. I got fed up and went to Mathematica. MathCAD was later bought by PTC, and further went to shit. It amuses me ho…

MathCAD went to shit in about 2000 That would be a couple of years after I last used it. What a shame, at least I have those memories. Really loved how low-friction working with it was, and how trivial it was to generate high quality documents or reports, there was no "write it up" stage, you were done basically as soon as you had done the actual work. Then just print and go! Still can't really do that with Jupyter.

Yes, it was a good run. Now I remember specifically:

Started using the DOS version in 1990 in college.

Bought version 3.1 in ~1992, Windows version running it under OS/2. Could afford to buy it then, even as a student. Ah, OS/2 was also good.

It was I had a home license which I dropped. I still had a work license, but then some sales twerp said I was using a pirated license at work, and wanted to speak to my IT department. This is a >100k employee company, so serious consequences for software piracy. I faxed them my valid license certificate, and told them to stuff it, dropping my renewal, and buying Mathematica.

Now I see it's Mathcad Prime, a subscription based license, for $800 year. They were purchsed by PTC several years ago, which only made things worse.

This is textbook case of screwing over your user base, cha$ing corporate accounts. I don't know anyone who uses it now. Many people used it back in the day.

Sad thing is I don't even use Mathematica much. Too painfull most of the time. I don't like the interface, but it's better than browser based Jupyter. Just forget trying to install Jupyter and dependancies on a corporate managed Windows computer. I just use papaer and calculator. I swear things have regressed from 20 years ago. At least I can still buy a new RPN calculator.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#99

Post by ChrisRackauckas » Sun, Apr 15, 2018, 3:19 PM UTC

gaius wrote: post #38

cs702 wrote: post #2

This is spot-on: "Membership in an open source community is like membership in the community of science. There is a straightforward process for finding a true answer to any question. People disagree in public conversations. They must explain clearly and listen to those who response with equal clarity. Members of the community pay more attention to those who have been right in the past, and to those who enhance their…

Membership in an open source community is like membership in the community of science. There is a straightforward process for finding a true answer to any question Oh please. Dare to ask what is the best of anything and prepare for an epic flame war.

Well, I asked the author for the data that led him to believe that the Julia community is "monopolistic", and he got mad and blocked me instead of linking to anything...

https://twitter.com/ChrisRackauckas/status/98552939470474035...

So your remark is pretty spot on.

Re: Jupyter, Mathematica, and the Future of the Research Paper

#100

Post by tincholio » Sun, Apr 15, 2018, 3:29 PM UTC

abhirag wrote: post #93

tincholio wrote: post #88

Earlier quoted context omitted.

You saved me a whole lot of typing! Thanks :) I'd add that you can also benefit from other aspects of org, such as project management functionality, outline editing, table editing, tables with formulas, direct git integration via Magit, etc., etc. I agree that getting people on Emacs is a non-minor issue, but using something like Spacemacs with CUA-mode enabled could go a long way towards acceptance.

I would love discussing the literate ray tracer I have been working on and the literate programming workflow I have made for it. Couldn't find your email in the profile, mine is abhirag@outlook.com, would be great to discuss if I can improve my workflow further :)

Just sent you a mail (check spam folder if you don't see it, it comes from my own domain, and sometimes it's dumped with the spam)