Why is Clojure so slow? • Hacker News Board

Re: Why is Clojure so slow?

#61

Post by igouy » Tue, Jul 10, 2012, 4:09 PM UTC

gizzlon wrote: post #38

lukev wrote: post #29

Earlier quoted context omitted.

And about twice as fast as Erlang, 5x faster than Ruby, Python or PHP and 10x faster than Perl.[1] It's all relative. [1] http://shootout.alioth.debian.org/u64q/which-programming-lan...

While interesting, those benchmarks are only useful if you implement something remotely similar to the actual benchmarks. Just glazing over them, they seem to be "unfairly" targeted at low-level languages. That said, they are fun to look at. I just had a wtf-moment looking at this: http://shootout.alioth.debian.org/u32/performance.php?test=n... ~20 seconds vs ~20 minutes??

1) those benchmarks are only useful if...

Any benchmark is only useful if...

http://shootout.alioth.debian.org/dont-jump-to-conclusions.p...

2) "unfairly" targeted at low-level languages

Don't make "unfair" accusations -- say why you think that.

3) ~20 seconds vs ~20 minutes??

Did you mean vs ~20 hours?

Re: Why is Clojure so slow?

#62

Post by ww520 » Tue, Jul 10, 2012, 4:10 PM UTC

bitcracker wrote: post #40

It's an implementation problem of Java. I never understood why the JVM folks didn't get along to develop a JIT cache. That means the first time I start a Java program it would run normally slow. But from the second run on it would use the native cache and run immediately fast with native performance. That would eliminate many performance problems of Java. I know that there already is a solution which uses a Java serv…

Did you read the article? The majority of the Clojure startup time is spent on initializing the Clojure runtime.

"spends 95% of the startup-time loading the clojure.core namespace (the clojure.lang.RT class in particular) and filling out all the metadata/docstrings etc for the methods. This process stresses the GC quite a bit, some 130k objects are allocated and 90k free-d during multiple invokes of the GC (3-6 times), the building up of meta data is one big source of this massive object churn."

Re: Why is Clojure so slow?

#63

Post by ithkuil » Tue, Jul 10, 2012, 4:21 PM UTC

CountHackulus wrote: post #58

bitcracker wrote: post #40

It's an implementation problem of Java. I never understood why the JVM folks didn't get along to develop a JIT cache. That means the first time I start a Java program it would run normally slow. But from the second run on it would use the native cache and run immediately fast with native performance. That would eliminate many performance problems of Java. I know that there already is a solution which uses a Java serv…

That "JIT cache" you're talking about already exists. It's known as AOT, or "Ahead of Time" compilation. I forget how you enable it, but it's there.

if you mean clojure AOT, it precompiles clojure to jvm bytecode. Usually the term JIT in this context is used to describe the native code generated by the VM on the flight, not the on the flight VM bytecode generation performed by a higher level language like clojure.

EDIT: sorry, probably your referred to http://publib.boulder.ibm.com/infocenter/java7sdk/v7r0/topic...

Re: Why is Clojure so slow?

#64

Post by igouy » Tue, Jul 10, 2012, 4:33 PM UTC

bad_user wrote: post #41

lukev wrote: post #29

Earlier quoted context omitted.

And about twice as fast as Erlang, 5x faster than Ruby, Python or PHP and 10x faster than Perl.[1] It's all relative. [1] http://shootout.alioth.debian.org/u64q/which-programming-lan...

Those benchmarks are fairly useless, because it doesn't compare idiomatic usage of those languages, but rather the capability of those languages to drop to low-level primitives and libraries. I did an experiment once, testing out a simple web-service on the JVM and Scala (with Scalatra) yields the same performance as Java (with Jax-RS), while Clojure (with Noir) is only 2x to 3x as slow and JRuby (Sinatra) is only 4x…

>>Those benchmarks are fairly useless, because it doesn't compare idiomatic usage of those languages...You seem to be complaining that the programmers were not forced to write slow programs :-)

>>I did an experiment once...And you used libraries.

Re: Why is Clojure so slow?

#65

Post by wanderingmarker » Tue, Jul 10, 2012, 4:35 PM UTC

To Clojure-curious people: please ignore this article. It is misinformed.

Others on this thread have pointed out that the Alioth benchmark takes startup time into account. Yes, this imposes a startup penalty on Clojure.

More importantly, the implementations of each individual benchmark vary significantly in performance quality. High-performance Clojure requires a couple of tricks in type hinting, using unchecked arithmetic, and preferring native Java arrays.

I looked at a couple of the benchmarks, and the mandelbrot example uses those tricks. Notice the performance there: http://shootout.alioth.debian.org/u64/performance.php?test=m...

Notice that Java 7 and gcc run at about the same speed, and Clojure is only about 2.2x slower (than C). Scala is about 1.9x slower (than C). gcc is about 1.5x slower than Intel Fortran.

To make the benchmark more fair: (1) all timings should disregard startup time; and (2) all JVM languages should have the opportunity to run the benchmark a few thousand times before timing it. Otherwise, it measures JVM startup time, and then it measures how long it takes the JIT to achieve maximum optimization. By comparison, the C and Fortran code runs at full speed almost out of the gate.

All-in-all, considering that Clojure is an extremely high-level language I consider its performance impressive [1]. Yes, the inner loops need to be coded in a slightly un-idiomatic manner, but you can do all this in the comfort of your REPL, which makes the process of making optimizations reasonably painless.

[1] Don't forget to scroll down the mandelbrot results and look at the stellar performance of other popular high-level languages.

Re: Why is Clojure so slow?

#66

Post by technomancy » Tue, Jul 10, 2012, 4:39 PM UTC

gaius wrote: post #43

bad_user wrote: post #39

Earlier quoted context omitted.

Slow/fast is a subjective metric, however in case of a language like Clojure you have to pay the price for convenience in the form of a performance tax when compared to Java or Scala. This may be acceptable, depends on where you're coming from and what you're needs are. For instance if you are coming from Ruby or Python because you want much better performance and you think the JVM is awesome for that, then Clojure m…

Tax is the wrong word... It's not the case that the Clojure (the language, the developers, etc) benefits from slow startup. Fine is probably more accurate.

Actually part of the reason startup is slow is that the JVM's JIT compiler doesn't optimize its compilations until it's determined that a given method is in a hotspot that would benefit from the optimization. Because it relies on runtime information, it's able to perform much better-informed optimizations than it would with a fully up-front compiler, but the price you pay is that the optimizations are deferred.

So in this case tax is exactly the right word.

Re: Why is Clojure so slow?

#67

Post by pjmlp » Tue, Jul 10, 2012, 4:45 PM UTC

bad_user wrote: post #48

cmkrnl wrote: post #20

Earlier quoted context omitted.

For my part, I don't trust Oracle to be a good steward for Java. Who knows - time will tell. It's not all gloom and doom though; Attila Szegedi is at Oracle now, working on Nashorn - something exciting for Java 8.

Unfortunately for Oracle, Java represents a genie that's out of the metaphorical bottle. They have the privilege to be its benevolent steward, however if they keep pushing for control in the face of its community, they'll lose whatever control they have left. The recent lawsuit kind of highlights that. Google is riding on years of development and refinement of Java IDEs and on mountains of available open-source libra…

What I miss in Java is the ability to compile directly to native code as part of the official SDK, instead of using expensive third party compilers.

The problem I see with Oracle is that if they push too much the community, companies might abandon it the same way they did with Delphi when Borland did too many mistakes. On the other hand, I can speak from my experience in the enterprise world, corporations love Oracle.

Re: Why is Clojure so slow?

#68

Post by Retric » Tue, Jul 10, 2012, 5:02 PM UTC

fauigerzigerk wrote: post #51

rbanffy wrote: post #9

How slow is slow? If it takes me 3 months to deliver a given program in Clojure and 6 to deliver its Java equivalent, the Clojure one already has 3 months of lead. Assuming the Java one is twice as fast, it'll take 45 days to catch up. Development time is expensive, computers are cheap and get twice as fast every year or so. While a long startup time is annoying, it can certainly be optimized out if someone focuses e…

That 45 day head start must be divided by the number of users running the program though.

Each of those users would be running it 45 days sooner so that cancels out.

Re: Why is Clojure so slow?

#69

Post by Jach » Tue, Jul 10, 2012, 5:04 PM UTC

gizzlon wrote: post #38

lukev wrote: post #29

Earlier quoted context omitted.

And about twice as fast as Erlang, 5x faster than Ruby, Python or PHP and 10x faster than Perl.[1] It's all relative. [1] http://shootout.alioth.debian.org/u64q/which-programming-lan...

While interesting, those benchmarks are only useful if you implement something remotely similar to the actual benchmarks. Just glazing over them, they seem to be "unfairly" targeted at low-level languages. That said, they are fun to look at. I just had a wtf-moment looking at this: http://shootout.alioth.debian.org/u32/performance.php?test=n... ~20 seconds vs ~20 minutes??

This sums up my feelings for the benchmarks:

http://shootout.alioth.debian.org/u64q/benchmark.php?test=fa...

Note that the "alternative" Lisp SBCL and Java 7 programs both outperform Fortran.

Of course I agree with you on wtf-moments. WTF makes Ruby take an hour, and SBCL take 10 seconds? That's two orders of magnitude! But no matter, I imagine a more clever Ruby programmer could reduce that, or just call out to a native library.

Re: Why is Clojure so slow?

#70

Post by fauigerzigerk » Tue, Jul 10, 2012, 5:09 PM UTC

Retric wrote: post #68

fauigerzigerk wrote: post #51

Earlier quoted context omitted.

That 45 day head start must be divided by the number of users running the program though.

Each of those users would be running it 45 days sooner so that cancels out.

No it doesn't. Development time is a one off (for a given feature set). Usage is recurring, so savings in running time catch up and eventually dwarf development time.

But I don't think this calculation makes much sense in the first place. It's simply not that linear and depends on many other things, for instance whether it's a throughput or response time problem, the relative value being the first to market versus being the best, etc.