Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computational reproducibility: IPython in the age of data-driven journalism

blog.fperez.org

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Computational reproducibility: IPython in the age of data-driven journalism

#11

Post by mistercow »

datr wrote: post #6
mistercow wrote: post #3

Since CoffeeScript started supporting literate programming, I've enjoyed dabbling with it quite a lot. I wrote a program to generate a recipe for a variant on the "soylent" concept, using numeric.js to solve and optimize for various constraints based on a target nutritional profile. The goal of that was rather close to what TFA is discussing; I had previously computed the recipe by hand, realized later that my calcul…

Documentation generator templates which split the comments and the code help with the scope problem of solution 1. E.g.: http://reload.github.io/phing-drupal-template/ The trade off being that the vertical spacing of the code is now inconsistent.

That does certainly help in terms of making the documentation readable, but it doesn't help the readability of the source code.

Re: Computational reproducibility: IPython in the age of data-driven journalism

#12

Post by dschep »

minopret wrote: post #2

Sage (sagemath.org) provides symbolic computing as well as numerical computing. Symbolic computing makes it easier to explain the equations behind the bottom-line results or diagrams and make that mathematical procedure open to computer-assisted experimentation. Sage is built on top of IPython. It is freely available both for download and as a web application (sagenb.org).

Is there any real benefit over IPython + SymPy?

Re: Computational reproducibility: IPython in the age of data-driven journalism

#13

Post by rogerallen »

minopret wrote: post #2

Sage (sagemath.org) provides symbolic computing as well as numerical computing. Symbolic computing makes it easier to explain the equations behind the bottom-line results or diagrams and make that mathematical procedure open to computer-assisted experimentation. Sage is built on top of IPython. It is freely available both for download and as a web application (sagenb.org).

This helped me understand the salient differences http://blog.fperez.org/2012/01/ipython-notebook-historical.h...

Re: Computational reproducibility: IPython in the age of data-driven journalism

#14

Post by eru »

mistercow wrote: post #8
jasongrout wrote: post #4

Earlier quoted context omitted.

Would it make a difference if your editor had better support for these sorts of things? For example, collapsed comments, where its shows only the first line, might help a lot. Then you could put a summary in the first line and elaborate in a paragraph that would be hidden by default. Or what if the editor supported such comments in the margin, like the LibreOffice or Word review comment features?

The problem really isn't that the description is too long, though. The problem is that you have to jump back and forth to see what part of the explanation applies to what part of the code. I'm also wary of making code readability too dependent on a specific editor The other problem with using numeric footnotes is that any time you change the code, you have to laboriously renumber the comments. That part really could…

Yes, traditional literate programming is a bit more powerful than what you get with CoffeeScript.

Have you tried reading the examples at http://www.literateprogramming.com/ ?

Re: Computational reproducibility: IPython in the age of data-driven journalism

#16

Post by minopret »

dschep wrote: post #12
minopret wrote: post #2

Sage (sagemath.org) provides symbolic computing as well as numerical computing. Symbolic computing makes it easier to explain the equations behind the bottom-line results or diagrams and make that mathematical procedure open to computer-assisted experimentation. Sage is built on top of IPython. It is freely available both for download and as a web application (sagenb.org).

Is there any real benefit over IPython + SymPy?

If you're happy with SymPy, then Sage doesn't really give you anything. That is, it gives you SymPy, which you already had.

Sage is many things intelligently integrated, of which SymPy is just one example. Superficially Sage is a set of command-line and web user interfaces with a preprocessor on top of the Python programming language. The deeper part of Sage is a set of libraries that tie together many high-quality tried-and-true open-source packages by providing a unified API and object model (theoretically based in category theory, I think), plus a set of package-specific APIs for when those are more desirable. The packages include Maxima as the main tool for symbolic computation, SymPy as another, NetworkX for graph-theoretic operations, PARI/GP for number theory, and many more: http://www.sagemath.org/links-components.html

Re: Computational reproducibility: IPython in the age of data-driven journalism

#17

Post by arpineh »

dr_doom wrote: post #10
arpineh wrote: post #5

The stuff you can do with current IPython notebook is already amazing and immensely useful, as the examples mentioned in the posts demonstrate. Browser is not a good place to manipulate gigabytes of data (like Python), but it is the easieast to use, most versatile rendering engine for visualizations. I keep envisioning a marriage of Light Table with IPython. If this post is any indication of Light Table's prowess, it…

Have you checked out Continuum's Wakari project.[1] It is essentially IPython with an online ide. http://continuum.io/wakari.html

I first mistook this for their other in browser data analysis environment: https://github.com/ContinuumIO/bokehjs

I have yet to test this, since it looks like early days yet.

Wakari is a cloud service and somewhat expensive for my needs. Though its hard to say how big an instance my data would actually require. If my Pandas experimentations don't pan out, I might have to give it a whirl.

Post reply on HN