Clojure on Hadoop: A New Hope
blog.factual.com
Clojure on Hadoop: A New Hope
1–10 of 29 posts
Re: Clojure on Hadoop: A New Hope
#2This is an entertaining comparison of 8 different map-reduce languages for Hadoop, including a flame-tastic take on Cascalog/Clojure: http://www.dataspora.com/2011/04/pigs-bees-and-elephants-a-c...
Re: Clojure on Hadoop: A New Hope
#3This is an entertaining comparison of 8 different map-reduce languages for Hadoop, including a flame-tastic take on Cascalog/Clojure: http://www.dataspora.com/2011/04/pigs-bees-and-elephants-a-c...
That's not flame-tastic. It's just inaccurate.
Re: Clojure on Hadoop: A New Hope
#4This is an entertaining comparison of 8 different map-reduce languages for Hadoop, including a flame-tastic take on Cascalog/Clojure: http://www.dataspora.com/2011/04/pigs-bees-and-elephants-a-c...
That's not flame-tastic. It's just inaccurate.
well, i'd agree it's "entertaining"! had fun reading that, thanks.
Re: Clojure on Hadoop: A New Hope
#5This is an entertaining comparison of 8 different map-reduce languages for Hadoop, including a flame-tastic take on Cascalog/Clojure: http://www.dataspora.com/2011/04/pigs-bees-and-elephants-a-c...
That's not flame-tastic. It's just inaccurate.
As someone who would like to start experimenting with Hadoop in near future, I would appreciate it a lot if you could elaborate on that or point me in the direction of a better comparison.
Re: Clojure on Hadoop: A New Hope
#6Earlier quoted context omitted.
That's not flame-tastic. It's just inaccurate.
As someone who would like to start experimenting with Hadoop in near future, I would appreciate it a lot if you could elaborate on that or point me in the direction of a better comparison.
Personally, I dislike LISP odd syntax, the widespread use of side effects in
a functional language and the poor abstraction that lists represent over RAM,
from a performance point of view — indeed LISP variants often add additional
data structures, somehow negating the “LIS” part of the language. In the
specific case of Clojure, the fact that a compiled language is compiled into
an interpreted one, JVM bytecode, combining a slow dev cycle with suboptimal
performance, makes me think Clojure users must be glutton for punishment.
Clojure has sophisticated state management. So much for widespread use of side effects.
Clojure has high performance data structure implementations tuned for modern hardware. So much for performance.
Clojure is compiled on the fly at the REPL and the JVM is one the fastest runtimes out there. So much for slow dev cycle.
You may not like the syntax, but boy is that Hadoop query short and sweet.
Re: Clojure on Hadoop: A New Hope
#7My experience with Hadoop tells me that its great for all counting tasks. Makes a lot of sense that it was designed at Google to construct posting lists for their search index. Beyond this sweet spot, it gets really tricky to map your solution to a map-reduce task. The programmer has to rely more on the fact that map/reduce are java black boxes to express everything he needs. Hadoop's big victory is the scale it operates on.
Are people here waiting for a "SQL-like-declarative-query-language + Hadoop" hybrid, which lets one write declarative processes to run on large amounts of data? Academia seems to be very motivated to produce something like this.
Re: Clojure on Hadoop: A New Hope
#8Earlier quoted context omitted.
As someone who would like to start experimenting with Hadoop in near future, I would appreciate it a lot if you could elaborate on that or point me in the direction of a better comparison.
Personally, I dislike LISP odd syntax, the widespread use of side effects in a functional language and the poor abstraction that lists represent over RAM, from a performance point of view — indeed LISP variants often add additional data structures, somehow negating the “LIS” part of the language. In the specific case of Clojure, the fact that a compiled language is compiled into an interpreted one, JVM bytecode, comb…
Yeah it's almost hilarious how every critic he is aiming at LISP is fixed in Clojure, where immutable is the default and vector the data structure of choice. Also he clearly doesn't understand the subtleties/differences between compilation and interpretation, and how the two can interlace.
Re: Clojure on Hadoop: A New Hope
#9My experience with Hadoop tells me that its great for all counting tasks. Makes a lot of sense that it was designed at Google to construct posting lists for their search index. Beyond this sweet spot, it gets really tricky to map your solution to a map-reduce task. The programmer has to rely more on the fact that map/reduce are java black boxes to express everything he needs. Hadoop's big victory is the scale it oper…
I work on Streams at IBM Research. Our solution to handling large amounts of data is called stream programming, and its more general than MapReduce. For a sample of what our language looks like, check out this example: http://publib.boulder.ibm.com/infocenter/streams/v2r0/index.... And a pdf version of the same: http://publib.boulder.ibm.com/infocenter/streams/v2r0/topic/...
Re: Clojure on Hadoop: A New Hope
#10This is an entertaining comparison of 8 different map-reduce languages for Hadoop, including a flame-tastic take on Cascalog/Clojure: http://www.dataspora.com/2011/04/pigs-bees-and-elephants-a-c...
"LISP has been around some 50-odd years without taking off despite several attempts at its revival... I suspect something is wrong with it"
I laughed out loud at that.