Live data from Hacker News

Web Framework Benchmarks

techempower.com

381–390 of 415 posts

Re: Web Framework Benchmarks

#381
post #322

Earlier quoted context omitted.

Play is not running on Resin. Did we accidentally say that somewhere? If so, we'll correct it.

Here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/mast... The tests were run with: Java OpenJDK 1.7.0_09 Resin 4.0.34 Play 2.1.0

Whoops! Fixing that up presently.

Thanks for the catch.

Re: Web Framework Benchmarks

#382

Since a few people have asked about Haskell, here are some Haskell benchmarks in comparison: http://www.yesodweb.com/blog/2011/03/preliminary-warp-cross-...

Still, it would be nice to have these results independently verified and put alongside all the rest. What do you think, bhauer? :)

Our to-do list is getting unwieldy. :) We would love to get this added, though.

Any chance a Haskell expert could create a test and submit it as a pull request?

Re: Web Framework Benchmarks

#383
These sort of "benchmarks" are so subjective that they really don't provide any value. Looking at the framework lineup, a lot of the frameworks at the bottom have so many (tightly coupled) features (which also lead to bloat, like excessive middleware layers) that they're obviously not going to perform anywhere near as well as some of the barebones frameworks at the top. Neat post, but yeah, zero value.

Re: Web Framework Benchmarks

#384
It's interesting to note the time per request when you invert these. Even some of the most drastic differences (like Spring vs. Django) turn out to be the difference of a millisecond or two. That's pocket change in terms of user experience.

Re: Web Framework Benchmarks

#385

Earlier quoted context omitted.

But Play's trivial JSON test was much slower than Netty's.

That's because they inexplicably use the Jackson library for the simple test, rather than Play's built in JSON support (they use the built-in JSON for the other benchmarks).

Both Netty and Play use Jackson though one the Netty version uses a single ObjectMapper and the Play version uses a new ObjectNode per request (created through Play's Json library).

Re: Web Framework Benchmarks

#386
post #382

Earlier quoted context omitted.

Still, it would be nice to have these results independently verified and put alongside all the rest. What do you think, bhauer? :)

Our to-do list is getting unwieldy. :) We would love to get this added, though. Any chance a Haskell expert could create a test and submit it as a pull request?

It sure sounds like it is!

I'm a far cry from a Haskell expert unfortunately, so I'll have to leave it to someone else.

Re: Web Framework Benchmarks

#387
post #171

Earlier quoted context omitted.

What's wrong with that guy?

He's a fantastic software engineer but he is _very_ abrasive. I've read some of his posts claiming he would fight some other developer in person at a conference if he steps up. He would rent out a ring and he would put his yellow belt to practice. Really, I'm not making this up. He sounds like a jerk to work with.

He's just a bit too honest and direct for the average American.

From that perspective at least, I expect people like Zed to do great in Northern Europe. There's a reason Americans think DHH is rude. He isn't; he's just Danish.

Re: Web Framework Benchmarks

#388

> This exercise aims to provide a "baseline" for performance across the variety of frameworks. By baseline we mean the starting point, from which any real-world application's performance can only get worse. I disagree with the implication here (that this is a good point for comparison because "real-world application's performance can only get worse."). Yes it can only get worse but how much worse (per unit of "featur…

This is completely loaded. Your implication is that the only viable test is a test which exercises all of the functionality of the most feature rich framework. How would that be a)viable and b) meaningful? We know that there is a set of common features and the benchmarks goal is to test least common denominator stuff on the networks. Authentication and portability are not LCD. The argument that they are is capricious…

I meant to suggest that comparing php-raw to Rails is apples & oranges- not "you must benchmark in a way that benefits larger frameworks", just "please acknowledge that LCD tests like this inherently cast Railsy frameworks in a bad light."

It's like condemning a swiss army knife because it's not as efficient as a fixed blade at cutting apples. Well yeah that's true, but what about when you need to screw a screw or pull a cork? One is a multitool, it doesn't make sense to compare it to a specialized tool unless all you plan to do is cut apples.

Re: Web Framework Benchmarks

#389

Earlier quoted context omitted.

But Play's trivial JSON test was much slower than Netty's.

That's because they inexplicably use the Jackson library for the simple test, rather than Play's built in JSON support (they use the built-in JSON for the other benchmarks).

No, they use Play's JSON lib. It's kind of a moot point because Play's lib is in fact a wrapper for Jackson.

Here's the source: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

So the question stands: If Play & Netty are using the same JSON serialization code, why is Play seven times slower?

Re: Web Framework Benchmarks

#390
post #309
post #240

Earlier quoted context omitted.

Please make sure you're setting higher GC limits for the Ruby tests. Ruby's defaults are awful for a framework, and result in a LOT of GC thrash. It's not uncommon to see an order of magnitude improvement in performance when they're tuned properly. (edit: I'll just send a pull request, I found the setup file!) Something else you might consider is the OJ gem rather than just the stock Ruby json gem. The latter is noto…

> make sure you're setting higher GC limits for the Ruby tests Could you elaborate on this, or point me in the right direction? I'm learning Rails and curious.

For anyone still lurking, this user replied to me via email:

Ruby allocates heaps for its objects, and sets GC thresholds based on those heap sizes. Ruby allows you to change those settings via environment variables, which means that you can end up doing fewer allocations and less aggressive GC, which makes sense when using a full framework like Rails, which is going to allocate a lot of objects.

There's a more complete answer here to get you started: http://stackoverflow.com/questions/13387664/ruby-gc-executio...

Post reply on HN