Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Zeppelin – Interactive analytics on Spark

zeppelin.incubator.apache.org

Post reply on HN
Print view View original

1–10 of 35 posts

Re: Apache Zeppelin – Interactive analytics on Spark

#2

Post by rkrzr »

Interesting. I wonder how this compares to an IPython notebook?

While I haven't tried it, I imagine that you can use pyspark from an IPython notebook and have much of the same functionality as with Apache Zeppelin. Makes me wonder if it wouldn't have been better to just extend IPython notebooks instead of starting a new project.

Edit: How to run PySpark from an IPython notebook[1]

[1] http://ramhiser.com/2015/02/01/configuring-ipython-notebook-...

Re: Apache Zeppelin – Interactive analytics on Spark

#3

Post by sshb »

rkrzr wrote: post #2

Interesting. I wonder how this compares to an IPython notebook? While I haven't tried it, I imagine that you can use pyspark from an IPython notebook and have much of the same functionality as with Apache Zeppelin. Makes me wonder if it wouldn't have been better to just extend IPython notebooks instead of starting a new project. Edit: How to run PySpark from an IPython notebook[1] [1] http://ramhiser.com/2015/02/01/c…

Also IPython is moving towards splitting the notebook part(Jupyter) and python part(IPython), so you can run any language in notebook (like IHaskell).

Details are in IPython changelog:

> 3.x will be the last monolithic release of IPython, as the next release cycle will see the growing project split into its Python-specific and language-agnostic components. Language-agnostic projects (notebook, qtconsole, etc.) will move under the umbrella of the new Project Jupyter name

https://jupyter.org/

http://ipython.org/ipython-doc/3/whatsnew/version3.html

Re: Apache Zeppelin – Interactive analytics on Spark

#4

Post by threeseed »

rkrzr wrote: post #2

Interesting. I wonder how this compares to an IPython notebook? While I haven't tried it, I imagine that you can use pyspark from an IPython notebook and have much of the same functionality as with Apache Zeppelin. Makes me wonder if it wouldn't have been better to just extend IPython notebooks instead of starting a new project. Edit: How to run PySpark from an IPython notebook[1] [1] http://ramhiser.com/2015/02/01/c…

Hilarious that i've tried this one but not the IPython one.

It's actually quite nice and very promising but still a little raw and rough around the edges. It's just great to see an open source project to compete with the commercial ones that seem to be popping up all over the place. And it's very modular so it could be used for more than just Spark based jobs.

Re: Apache Zeppelin – Interactive analytics on Spark

#7

Post by mhuffman »

spot wrote: post #5

Notebook interfaces are proliferating. Here's the one I work on: http://BeakerNotebook.com Note that we are hiring, especially front-end engineers. Do open source fulltime in NYC.

This is really very nice! For me, seem like it might scratch an itch, on which I have had to rely on RStudio (which is also awesome for iterative exploration and display, but a "heavy" context switch if R is not your favorite language ).

Re: Apache Zeppelin – Interactive analytics on Spark

#9

Post by rkrzr »

thebouv wrote: post #6

I must admit I now need to go figure out what a "notebook" is in this context. I don't really get what Zeppelin -is-. But I'm curious.

Think of it as an interactive notebook where you can not only write down notes, but also enter data and code and interact with it. You can then draw nice graphs of the data and explore it interactively.

The nice thing about it is that it is all repeatable and that you can share it with other people/researchers.

Re: Apache Zeppelin – Interactive analytics on Spark

#10

Post by wcbeard10 »

I have no idea about Zeppelin's lineage, but looks like there's also Spark Notebook https://github.com/andypetrella/spark-notebook which more closely resembles the IPython notebook. I'd love to hear an explanation of the differences between all of these notebooks.

Can't wait to see if the Jupyter split will contribute to a consolidation or proliferation in the notebook-verse...

Post reply on HN