Why is that Clojure is consistently slower in the Alioth Benchmarks game [1], but Compojure is consistently faster the Play-scala?
[1] http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
Web Framework Benchmarks Round 3
11–20 of 30 posts
Re: Web Framework Benchmarks Round 3
#12Thanks to techempower for putting this together. It's fantastic for me as the author of Phreeze to see how my framework stacks up, I've always been curious. One strange thing is that Phreeze rocks on the multi-query test on the EC2 instance but on dedicated hardware does poorly. Would anyone have a clue why that would be? At the PHP level I have to admit I don't really factor in performance tuning for specific hardwa…
Hi jakejake, thanks for the kind words and for contributing your Phreeze test. It's looking real good, and I hope to have at least language color-coding in the next round so you can more easily see how it compares to your PHP peers.
As for your question, I'm not certain why the physical hardware turned in a lower score than EC2. That suggests a configuration problem since we know the physical hardware is in fact quite a bit higher-performance. Pat (pfalls) may be able to find some time to help you diagnose it further.
Have you had a chance to benchmark Phreeze on some of your own hardware to fine-tune its configuration?
Re: Web Framework Benchmarks Round 3
#13Where is all the attention that happened for the first two rounds? (I'm really hoping someone will explain why Go's database results are so slow, and what can be done to improve that.)
The most likely cause is in the mysql driver. Go provides an interface for drivers to implement, similar to how JDBC does in Java. One of the go developers, Brad Fitzpatrick, commented [1] that the actual code used by the benchmark looks OK. The discussion of these test results on golang-nuts [1] triggered a number of performance related pull requests [2] to the mysql driver used. Also of interest: https://code.googl…
Excellent, thanks much!
Re: Web Framework Benchmarks Round 3
#14Why is that Clojure is consistently slower in the Alioth Benchmarks game [1], but Compojure is consistently faster the Play-scala? [1] http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
Many variables are involved, but my conjecture is that it has more to do with the particulars of Compojure versus Play than it does Clojure versus Scala. For instance, take a look at the numbers put up by other Scala frameworks (Unfiltered, Lift, Scalatra). There's nothing intrinsically slow about Scala.
Re: Web Framework Benchmarks Round 3
#15This is the latest update to our benchmarking of web application frameworks and platforms. Since Round 2, we've had several pull requests. There is more Scala, Erlang, Lua, PHP, Java, Haskell, more everything! (Sorry, we've not yet received .NET/Mono pull requests.) Additionally, with the help of the author of Wrk, we've been able to change the methodology to use time-limited tests (1 minute) rather than request-limi…
I know all languages are using mysql, but mysql drivers are pretty poor for languages such as 'go' and 'python'. So it does make it quite unbalanced. I'd love to see this with Postgres.
I agree. We do plan to add Postgres tests in the future. I don't have an ETA for that, however.
Re: Web Framework Benchmarks Round 3
#16Thanks to techempower for putting this together. It's fantastic for me as the author of Phreeze to see how my framework stacks up, I've always been curious. One strange thing is that Phreeze rocks on the multi-query test on the EC2 instance but on dedicated hardware does poorly. Would anyone have a clue why that would be? At the PHP level I have to admit I don't really factor in performance tuning for specific hardwa…
Hi jakejake, thanks for the kind words and for contributing your Phreeze test. It's looking real good, and I hope to have at least language color-coding in the next round so you can more easily see how it compares to your PHP peers. As for your question, I'm not certain why the physical hardware turned in a lower score than EC2. That suggests a configuration problem since we know the physical hardware is in fact quit…
Thanks so much for the reply. I haven't been able to get the full suite running where it creates the environment and everything via the setup scripts - I just have tried to reproduce the Nginx environment and run tests manually. But I really would like to duplicate the whole scaffolding so I can see what may be going on. I have a crushing deadline in two weeks, then after that I'm going to devote some time to getting the full thing running.
Color coding per language would be fantastic too!
Re: Web Framework Benchmarks Round 3
#17This is the latest update to our benchmarking of web application frameworks and platforms. Since Round 2, we've had several pull requests. There is more Scala, Erlang, Lua, PHP, Java, Haskell, more everything! (Sorry, we've not yet received .NET/Mono pull requests.) Additionally, with the help of the author of Wrk, we've been able to change the methodology to use time-limited tests (1 minute) rather than request-limi…
what is meant by raw as in php-raw vs php
Re: Web Framework Benchmarks Round 3
#18This is the latest update to our benchmarking of web application frameworks and platforms. Since Round 2, we've had several pull requests. There is more Scala, Erlang, Lua, PHP, Java, Haskell, more everything! (Sorry, we've not yet received .NET/Mono pull requests.) Additionally, with the help of the author of Wrk, we've been able to change the methodology to use time-limited tests (1 minute) rather than request-limi…
what is meant by raw as in php-raw vs php
For most of the tests, we expect that an ORM or something ORM-like is used to work with the database. For example, the Ruby tests use ActiveRecord and many of the Java tests use Hibernate. We believe the use of an ORM to be conventional for most production sites.
The "raw" suffix indicates that an ORM is not used. This can give you an idea of the cost of using an ORM (or, in some cases, the cost of having more framework code in general).
The "servlet-raw" test uses raw JDBC to connect to and query the database. The "php-raw" test uses PHP's raw MySQL connectivity and no ORM. The "php" test with no "raw" suffix is using PHP ActiveRecord.
Re: Web Framework Benchmarks Round 3
#19Wow, thank you for including Lift, It's mind blowing to see Lift perform only marginally better than Ruby on Rails...I always thought it as a very performant framework...
Lift is optimized for maintaining persistent connections with users and maintaining their state for an easier programming model and probably performs well at that.
If your use case actually matches these benchmarks then yes, Lift would be a poor choice.
Re: Web Framework Benchmarks Round 3
#20Wow, thank you for including Lift, It's mind blowing to see Lift perform only marginally better than Ruby on Rails...I always thought it as a very performant framework...
From what I have seen Lift performs much better than Rails except in the benchmarks where there were errors running the tests.