Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Zeppelin – Interactive analytics on Spark

zeppelin.incubator.apache.org

Post reply on HN
Print view View original

11–20 of 35 posts

Re: Apache Zeppelin – Interactive analytics on Spark

#11

Post by 16bytes »

thebouv wrote: post #6

I must admit I now need to go figure out what a "notebook" is in this context. I don't really get what Zeppelin -is-. But I'm curious.

Notebook in this example is much like the notebook concept in ipython.

Basically, it is a literate program that allows you to intersperse code, documentation and visualizations in the same "file" called a notebook.

If you are familiar with how mathematica works, it's basically the same thing.

If you do data-science at all, or even get tired of working in excel, I recommend picking up ipython and matplotlib.

Re: Apache Zeppelin – Interactive analytics on Spark

#13

Post by samkone »

rkrzr wrote: post #2

Interesting. I wonder how this compares to an IPython notebook? While I haven't tried it, I imagine that you can use pyspark from an IPython notebook and have much of the same functionality as with Apache Zeppelin. Makes me wonder if it wouldn't have been better to just extend IPython notebooks instead of starting a new project. Edit: How to run PySpark from an IPython notebook[1] [1] http://ramhiser.com/2015/02/01/c…

Yes you can use IPython with Spark for Pyspark. Spark-notebook and Zeppelin, gives let's you use Scala with Spark,and support all of the other spark libraries. There are other projects like spark-kernel https://github.com/ibm-et/spark-kernel that let's you use IPython (Jupyter ) I guess, supporting spark's scala API.

Re: Apache Zeppelin – Interactive analytics on Spark

#14

Post by drincruz »

spot wrote: post #5

Notebook interfaces are proliferating. Here's the one I work on: http://BeakerNotebook.com Note that we are hiring, especially front-end engineers. Do open source fulltime in NYC.

Wow, now this looks useful! Ability to pass data between Python to R seamlessly? I'm putting this on my list to check out. Thanks!

Re: Apache Zeppelin – Interactive analytics on Spark

#15

Post by spot »

po84 wrote: post #12

The size and growth of the Jupyter / IPython ecosystem is its biggest draw for me. For example, there are roughly 125,000 IPython notebooks on GitHub as of today from which I can learn. http://nbviewer.ipython.org/gist/parente/facb555dfbae28e817e...

Beaker is mostly compatible, and has an import feature, so you can open and run an IPython notebook as a Beaker Notebook.

Re: Apache Zeppelin – Interactive analytics on Spark

#16

Post by ujjwal_wadhawan »

thebouv wrote: post #6

I must admit I now need to go figure out what a "notebook" is in this context. I don't really get what Zeppelin -is-. But I'm curious.

A good demo would be to watch https://www.youtube.com/watch?v=lO7LhVZrNwA. This was a real time demo at Spark Summit last year.

Note: The demo gets pretty interesting at https://youtu.be/lO7LhVZrNwA?t=36m50s. The presenter handles the situation very well afterwards !

Re: Apache Zeppelin – Interactive analytics on Spark

#17

Post by leemoonsoo »

rkrzr wrote: post #2

Interesting. I wonder how this compares to an IPython notebook? While I haven't tried it, I imagine that you can use pyspark from an IPython notebook and have much of the same functionality as with Apache Zeppelin. Makes me wonder if it wouldn't have been better to just extend IPython notebooks instead of starting a new project. Edit: How to run PySpark from an IPython notebook[1] [1] http://ramhiser.com/2015/02/01/c…

I'm one of committers of Apache Zeppelin (incubating). Zeppelin is inspired by iPython notebook and many other amazing softwares that has notebook interface.

I know iPython notebook has long history and large community, i really like it. Zeppelin is young, new project compare to iPython notebook.

Zeppelin and iPython notebook, they are opensource. iPython notebook is lead by IPython Development Team. Zeppelin is under Apache Software Foundation and it is being developed in Apache way, from copyrights, development process, decision making process to community development.

Zeppelin is focusing on providing analytical environment on top of Hadoop eco-system. I'm not sure about iPython's direction, but i don't think it's the same to Zeppelin.

I see many projects that has notebook interface. Not only iPython and Zeppelin, but also Databricks Cloud, Spark Notebook, Beaker and many others. I'm sure they all have their own advantages. Hope see all softwares are beloved by users.

Cheers

Re: Apache Zeppelin – Interactive analytics on Spark

#18

Post by nchammas »

ujjwal_wadhawan wrote: post #16
thebouv wrote: post #6

I must admit I now need to go figure out what a "notebook" is in this context. I don't really get what Zeppelin -is-. But I'm curious.

A good demo would be to watch https://www.youtube.com/watch?v=lO7LhVZrNwA . This was a real time demo at Spark Summit last year. Note: The demo gets pretty interesting at https://youtu.be/lO7LhVZrNwA?t=36m50s . The presenter handles the situation very well afterwards !

Just to be clear, though that video shows an example of what a notebook is in this context, that isn't a demo of Zeppelin. (It wasn't clear which you were referencing in your comment.)

That's a demo of Databricks Cloud [0], which is Databricks's product offering.

Among the many things it offers is an interactive notebook that looks like open source alternatives like Zeppelin.

[0] https://databricks.com/product/databricks-cloud

Re: Apache Zeppelin – Interactive analytics on Spark

#19

Post by mbrzusto »

Notebooks where all the rage in the 90's. I spent quite a few years (~15) with Mathematica and its "notebook". Of course, Maple, Mathcad and Matlab all had the same thing. At some point, I wanted to write more readable and modular code that others could use (and that I could reuse) so I switched to using and IDE for python, Java and C. My workflow enables exploratory data analysis as well as algorithm development. The huge advantage of IDE's vs. REPL's is that in the end, it's easier to write the quality of code that is ready to be shipped off to a production server. I have watched the "Return of the Notebook" with some trepidation: yes, it allows quick iteration and lower barrier to entry, but ultimately if your goal is to create a software product, learning good coding skills in an IDE is a much better path.

Re: Apache Zeppelin – Interactive analytics on Spark

#20

Post by leemoonsoo »

For someone who is interested, Here's some videos of Apache Zeppelin (Incubating)

https://www.youtube.com/watch?v=QdjZyOkcG_w

https://www.youtube.com/watch?v=_PQbVH_aO5E&feature=youtu.be

https://www.youtube.com/watch?v=cWuWUvWVLx4

https://www.youtube.com/watch?v=xU5TBS_MsAs

Post reply on HN