Web Framework Benchmarks • Hacker News Board

Re: Web Framework Benchmarks

#21

Post by goodwink » Thu, Mar 28, 2013, 2:34 PM UTC

One of the most interesting things this comparison brings out to me is not so much the differences between the various frameworks (although the differences between options on the same platform is definitely very useful information), but also the issue that few of us seem to think about these days: the cost of any of the frameworks above the bare standard library of the platform its hosted on.

Theres a consistant, considerable gap between their "raw" benchmarks (things like netty, node, plain php, etc.) and frameworks hosted on those same platforms. I think this is something we should keep in mind when we're tuning performance-sensitive portions of APIs and the like. We may actually need to revisit our framework choice and implement selected portions outside of it (just like ruby developers sometimes write performance-critical pieces of gems in C etc.) or optimize the framework further.

I'd like to crunch these numbers further to get a "framework optimization index" which would be the percentage slowdown or ratio of performance between the host platform and the performance of the framework on top of it. I might do this later if I get a chance.

Re: Web Framework Benchmarks

#22

Post by asadkn » Thu, Mar 28, 2013, 2:34 PM UTC

Kiro wrote: post #6

So this means I should stop using CakePHP?

Not really. Take this benchmark with a grain of salt unless they benchmark it again with APC at minimum. Nobody serious about a web app would run a PHP app without APC.

Similarly, relatively modern deployment standards (not really cutting-edge) like nginx and php-fpm OR apache 2.4 with worker mpm + php-fpm should be added to the mix.

Re: Web Framework Benchmarks

#23

Post by dkroy » Thu, Mar 28, 2013, 2:35 PM UTC

anuraj wrote: post #16

Kiro wrote: post #6

So this means I should stop using CakePHP?

If you care about performance, the thinner the framework, the better.

Doesn't matter if you compile it like Facebook had done with their php at one time when they needed to scale.

Re: Web Framework Benchmarks

#24

Post by bhauer » Thu, Mar 28, 2013, 2:35 PM UTC

anuraj wrote: post #14

Interesting - C# should have also been added. Java still rules the roost. If you should absolutely have a scripting language - old php with warts is better. More magic in the framework - more abstraction and indirection and code inefficiency - that can be the spoil port when it comes to performance.

We agree and want to get a C# test in there. It's among the top priorities for us.

Re: Web Framework Benchmarks

#25

Post by pfalls » Thu, Mar 28, 2013, 2:36 PM UTC

jondot wrote: post #17

pfalls - amazingly, I spent the last 2 days of my holiday doing the same thing for a future open source project. I was just stumped when I saw you guys did the same (could have saved me a couple of days!) I wanted to find the leanest Web framework on any kind of platform; but the difference from your approach - I already knew the kind of code that would run on it. I tested: Go, Java (servlet, dropwizard), Scala (scal…

Thanks for the comment!

This started out as a small exercise, that quickly ballooned because we were curious about every framework and platform. Obviously we had to stop somewhere, but we're very interested in adding more tests in the future. In fact, we're hoping the community will help us out as well!

Re: Web Framework Benchmarks

#26

Post by jondot » Thu, Mar 28, 2013, 2:36 PM UTC

goodwink wrote: post #21

One of the most interesting things this comparison brings out to me is not so much the differences between the various frameworks (although the differences between options on the same platform is definitely very useful information), but also the issue that few of us seem to think about these days: the cost of any of the frameworks above the bare standard library of the platform its hosted on. Theres a consistant, con…

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

Re: Web Framework Benchmarks

#27

Post by bhauer » Thu, Mar 28, 2013, 2:37 PM UTC

wldlyinaccurate wrote: post #18

Cake was an interesting choice for a PHP framework to test. I wonder why they didn't choose Symfony, which is arguably the leading PHP framework.

It was simply what we have seen most often in our experience. We've got Zend as a next target for PHP frameworks. We can also put Symfony on the list.

We'd also love to have anyone who has production experience with those to contribute a test for them. The test should be fairly quick to write.

Re: Web Framework Benchmarks

#28

Post by atonse » Thu, Mar 28, 2013, 2:37 PM UTC

I'm torn about this. On the one hand, while I've known my framework of choice (Rails) is slow, I didn't know how much slower it could be in the grand scheme of things. But on the other hand, I'm more shocked by the difference between EC2 and dedicated hardware (10x improvement with rails), and even 89 requests per second (20 query benchmark on EC2) is still a decent amount of traffic. (Plus this doesn't count any optimizations I would make anyway, like caching).

Either way good architectures usually optimize the high traffic or high CPU areas anyway away from a scripted language.

Thanks for the really informative post! Go seems to be a good balance as a high performance language without having to go back to my traumatic Java days.

Re: Web Framework Benchmarks

#29

Post by gruseom » Thu, Mar 28, 2013, 2:37 PM UTC

It would be very interesting to see OpenResty (Nginx + Lua) in there, since it's so different from other approaches.

Re: Web Framework Benchmarks

#30

Post by asadkn » Thu, Mar 28, 2013, 2:38 PM UTC

wldlyinaccurate wrote: post #18

Cake was an interesting choice for a PHP framework to test. I wonder why they didn't choose Symfony, which is arguably the leading PHP framework.

Symfony and Zend might perform even worse. At least Zend's bootstrapping is even heavier. The sheer number of files that have to be loaded from disk on each hit is appalling. Hence I mentioned in another comment the opcode cache (ex. APC) requirement and how every serious PHP app utilizes it.