Now Serving: Full-Text Sampler in IPython Notebook Format • Hacker News Board

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#21

Post by ptwobrussell » Fri, Oct 25, 2013, 4:50 PM UTC

sillysaurus2 wrote: post #19

ptwobrussell wrote: post #17

Earlier quoted context omitted.

There are some awesome notebooks out that demonstrate exactly that. Here are a couple of good links with combinatorial optimization and TSP: http://nbviewer.ipython.org/urls/raw.github.com/fonnesbeck/B... http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipy...

This is the first I've looked into IPython Notebook, so apologies for the newbie question: Is a notebook an html page with Python code snippets interspersed? Are these supposed to be interactive at all? Or do you have to copy the Python locally, then install some packages before you can finally run it? I guess I'm asking, what's the difference between an HTML page that shows a Python implementation of an algorithm vs…

The links that you're viewing are "read-only" version of the notebooks. The great thing about them if you install IPython Notebook and "run" them is that you are able to read the content, and then when you encounter code, you execute it interactively - kind of like Mathematica if you've ever used that?

There are some videos that show this in action. One that I put together is here if you'd like to see a quick demo of it as it (rather generically) relates to Mining the Social Web - https://vimeo.com/72871930

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#22

Post by mcburton » Fri, Oct 25, 2013, 4:56 PM UTC

sillysaurus2 wrote: post #19

ptwobrussell wrote: post #17

Earlier quoted context omitted.

There are some awesome notebooks out that demonstrate exactly that. Here are a couple of good links with combinatorial optimization and TSP: http://nbviewer.ipython.org/urls/raw.github.com/fonnesbeck/B... http://nbviewer.ipython.org/url/norvig.com/ipython/TSPv3.ipy...

This is the first I've looked into IPython Notebook, so apologies for the newbie question: Is a notebook an html page with Python code snippets interspersed? Are these supposed to be interactive at all? Or do you have to copy the Python locally, then install some packages before you can finally run it? I guess I'm asking, what's the difference between an HTML page that shows a Python implementation of an algorithm vs…

The IPython Notebook is part of the IPython interactive computing environment[1]. You must to install IPython locally(or in the cloud) and run the notebook web server to load and execute the notebook's code live in the browser. These are static HTML representations of the notebooks, think of them as previews, but you can download the JSON .ipynb file and load it up in your IPython Notebook environment and execute the code.

The benefit of the IPython Notebook is sharable JSON file and interactive REPL-like environment that blends code, results, and narrative prose.

[1] http://ipython.org/ipython-doc/stable/interactive/notebook.h...

edit: I should have refreshed the page before I wrote this redundant reply.

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#23

Post by julienchastang » Fri, Oct 25, 2013, 6:56 PM UTC

I work with scientists that love the IPython Notebook technology. Some claim the IP[y]: Notebook to be the best thing since the Mosaic web browser and the most important development in scientific computing in a decade. They are replacing a variety of tools (Perl, Matlab, R, etc.) with the notebook technology. I tend to agree, it is a revolutionary technology and the idea of executable papers is tantalizing. But I also see some limitations. Setting up a Python environment with all the necessary libraries can be a real pain in the neck even with technologies like pip. Also taking advantage of multicore parallelism is not trivial because of the Python Global Interpreter Lock. I have also worked in JVM environments where parallel computing is becoming significantly easier and I don't see that happening in Python anytime soon. I would love to be proven wrong, of course.

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#24

Post by carreau » Fri, Oct 25, 2013, 7:12 PM UTC

That's really amazing, I think I can speek for the all IPython team by saying that we are Really happy with this.

We've cross a few authors that were writing books for O’Reilly with IPython Notebooks in the last few month and we have plan to help convert those to (and from) many format. We would be extremely happy to work with people from O'Reilly to simplify the conversion process.

We also have plan to integrate with publication in academic world (see https://github.com/ipython/ipython/issues/4119) but we really need more people involved than the few of us (Core Team).

Even if IPython seem like a big complicated project, moving forward in area like document conversion is not that hard, but just needs lots of eyes and tester to catch all cases.

As for people wanting to convert notebook to other format than those currently supported, it is perfectly possible using nbconvert with template. We can spit the TeX that is just between begin/end document so you can include in your preferred dissertation template. It should be pretty easy to also just write a converter that export to asciidoc or doocbook.

For those interested of the idea behing how nbconvert I wrote this quite some time ago: http://nbviewer.ipython.org/urls/raw.github.com/Carreau/post... and https://github.com/ipython/ipython-in-depth Offer some example too.

Hope this will push some people to contribute, we'll always be happy do guide you in your first step.

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#25

Post by ptwobrussell » Fri, Oct 25, 2013, 7:26 PM UTC

julienchastang wrote: post #23

I work with scientists that love the IPython Notebook technology. Some claim the IP[y]: Notebook to be the best thing since the Mosaic web browser and the most important development in scientific computing in a decade. They are replacing a variety of tools (Perl, Matlab, R, etc.) with the notebook technology. I tend to agree, it is a revolutionary technology and the idea of executable papers is tantalizing. But I als…

I would totally agree on the first point about how much of a burden it can be to get any moderately complex Python environment up and running (whether IPython Notebook is part of the install or not.) Unfortunately, it's just not as easy as it should be yet.

IMHO, this is what makes a duo like Vagrant + IPython Notebook so appropriate for creating a minimum barrier to entry. In other words, what you really need is a VM that has IPython Notebook and all of the necessary dependencies installed that you just start up as a guest machine. Then, on your host, you point your browser to http://localhost:8888 and get to work with the the ipynb UI.

The approach I described is exactly what I did with Mining the Social Web 2E, and I elaborated on it a bit here: http://miningthesocialweb.com/2013/08/24/reflections-on-auth...

The cool thing about the way Vagrant works is that you can run a VM locally or you can run it in the cloud just as easily with the appropriate plugin.

There's a video that demonstrates how easy it can be to use Vagrant and IPython Notebook to get started here - https://vimeo.com/72383764

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#26

Post by ptwobrussell » Fri, Oct 25, 2013, 7:45 PM UTC

julienchastang wrote: post #23

I work with scientists that love the IPython Notebook technology. Some claim the IP[y]: Notebook to be the best thing since the Mosaic web browser and the most important development in scientific computing in a decade. They are replacing a variety of tools (Perl, Matlab, R, etc.) with the notebook technology. I tend to agree, it is a revolutionary technology and the idea of executable papers is tantalizing. But I als…

By the way, I really like the metaphor of describing IPython Notebook as "executable paper" - that's clever and concise.

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#27

Post by mcburton » Fri, Oct 25, 2013, 9:00 PM UTC

julienchastang wrote: post #23

I work with scientists that love the IPython Notebook technology. Some claim the IP[y]: Notebook to be the best thing since the Mosaic web browser and the most important development in scientific computing in a decade. They are replacing a variety of tools (Perl, Matlab, R, etc.) with the notebook technology. I tend to agree, it is a revolutionary technology and the idea of executable papers is tantalizing. But I als…

It should be noted that the IPython Notebook is being developed to support multiple backends. People are working on backends for Haskell[1], Scala [2], and Julia[3]. I'd love to see a Clojure backend.

All of this does not obviate the challenges of setting up IPython, but that could also be (somewhat)resolved via cloud services[4][5][6].

[1] http://gibiansky.github.io/IHaskell/

[2] https://github.com/mattpap/IScala

[3] https://github.com/JuliaLang/IJulia.jl

[4] http://www.windowsazure.com/en-us/develop/python/tutorials/i...

[5] https://www.wakari.io/

[6] http://blog.picloud.com/2012/12/23/introducing-the-picloud-n...

Re: Now Serving: Full-Text Sampler in IPython Notebook Format

#28

Post by IanOzsvald » Mon, Oct 28, 2013, 2:34 PM UTC

carreau wrote: post #24

That's really amazing, I think I can speek for the all IPython team by saying that we are Really happy with this. We've cross a few authors that were writing books for O’Reilly with IPython Notebooks in the last few month and we have plan to help convert those to (and from) many format. We would be extremely happy to work with people from O'Reilly to simplify the conversion process. We also have plan to integrate wit…

I'm working on an O'Reilly book using IPython Notebook, I'm basing my workflow on Olivier Grisel's script (with a couple of tiny fixes): https://github.com/ianozsvald/ipynbhelper which extracts the Notebook's 'markdown' blocks (which contain asciidoc which obviously won't render in the browser) & code blocks, these get exported as asciidoc for O'Reilly.

It is clunky but I'm hoping we'll get better control as nbconvert evolves, so we're experimenting with this approach.

Most of the code examples are not 'live', they're pasted in along with analysis results (the book is about high performance and parallel computing: http://shop.oreilly.com/product/0636920028963.do ), as lots of the examples are best run from a fresh VM.