Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks

techempower.com

Post reply on HN
Print view View original

151–160 of 415 posts

Re: Web Framework Benchmarks

#151

Post by pmahoney »

daenz wrote: post #142
xal wrote: post #135

What kind of server did you guys use for your rails test? Thin, Puma, Unicorn? Are you sure you ran it in production environment? Update: Looks like passenger in development mode. Good job you benchmarked a web server that no one uses wile reloading all code between requests. Update2: Ok it seems to run in production mode but still, passenger is not an idiomatic choice.

I'm not familiar with Rails. Could you explain what you mean in layman terms? Why is code being reloaded on every request by default?

In development mode, it's nice to be able edit ruby code, then hit reload in the browser to see it in action without restarting the whole app. Sure there are other ways to accomplish this, but reloading code is typically how it's done in Ruby.

Re: Web Framework Benchmarks

#152

Post by proland »

bhauer wrote: post #7
goodwink wrote: post #4

Earlier quoted context omitted.

I'd be interested to see performance for Vert.x on its other hosts (this is the JVM version, I believe).

Interesting point. You're correct, we've only tested Vert.x as a Java/JVM platform.

I agree, seeing vert.x with it's other language options would be interesting.

Re: Web Framework Benchmarks

#153

Post by pekk »

papsosouid wrote: post #141
pekk wrote: post #96

Numerous irregularities plus a strong vested interest in the JVM make me doubt they have given adequate shrift to Go, here. Given the amount of interest in Haskell and Yesod around here, it is strange that it is missing.

>Given the amount of interest in Haskell I see more people making uninformed "haskell sucks" posts than expressing interest in it. >and Yesod Really? Yesod is the anti-haskell haskell framework.

You and tikhonj are all over the place in here. If you were being downvoted into the gray, you would know. There is no shortage of people praising Haskell every day, this is what is in fashion today.

There are also regular posts about Yesod.

I conclude that you know perfectly well that Haskell and Yesod are regularly mentioned on HN, but find it inconvenient to have mentioned for some reason I do not fathom.

Re: Web Framework Benchmarks

#155

Post by rallison »

goodwink wrote: post #77
cacois wrote: post #42

Earlier quoted context omitted.

I'd love to see your framework optimization index. Honestly, all of this would be a wonderful thing to automate and put in a web app - a readily-accessible, up-to-date measure of the current performance of the state of the art in languages and frameworks. I bet it would really change some of the technology choices made.

Here's a quick version of the framework optimization index. Higher is better (ratio of framework performance to raw platform performance, multiplied by 100 for scale): Framework Framework Index Gemini 87.88 Vert.x 76.29 Express 68.85 Sinatra-Ruby 67.88 WebGo 51.08 Compojure 45.69 Rails-Ruby 31.75 Wicket 29.33 Rails-Jruby 20.09 Play 18.02 Sinatra-Jruby 15.96 Tapestry 13.57 Spring 13.48 Grails 7.11 Cake 1.17

In the same vein, I was curious to compare the max responses/second on dedicated hardware vs ec2 on a per framework basis. The following is percentage throughput of ec2 vs dedicated (in res/s):

cake 18.9% (312 vs 59)

compojure 12.1% (108588 vs 13135)

django 16.8% (6879 vs 1156)

express 16.9% (42867 vs 7258)

gemini 12.5% (202727 vs 25264)

go 13.3% (100948 vs 13472)

grails 7.1% (28995 vs 2045)

netty 18% (203970 vs 36717)

nodejs 15.6% (67491 vs 10541)

php 11.6% (43397 vs 5054)

play 20.6% (25164 vs 5181)

rack-jruby 15.6% (27874 vs 4336)

rack-ruby 22.7% (9513 vs 2164)

rails-jruby 22.7% (3841 vs 871)

rails-ruby 20.7% (3324 vs 687)

servlet 13.4% (213322 vs 28745)

sinatra-jruby 21.2% (3261 vs 692)

sinatra-ruby 22.2% (6619 vs 1469)

spring 7.1% (54679 vs 3874)

tapestry 5.2% (75002 vs 3901)

vertx 22.3% (125711 vs 28012)

webgo 13.5% (51091 vs 6881)

wicket 12.7% (66441 vs 8431)

wsgi 14.8% (21139 vs 3138)

I found it interesting that something like tapestry took a 20x slowdown when going from dedicated to ec2, while others only took ~5x slowdown.

Edit: To hopefully make it clearer what the percentages mean - if a framework is listed at 20%, this means that the framework served 1 request on ec2 for every 5 requests on dedicated hardware. 10% = 1 for every 10, and so on. So, higher percentage means a lower hit when going to ec2.

Disclosure: I am a colleague of the author of the article.

Re: Web Framework Benchmarks

#156

Post by josh-j »

bhauer wrote: post #109
pilgrim689 wrote: post #74

Earlier quoted context omitted.

ditto. I hear Warp (the server behind Yesod) is a beast.

I'm not familiar with Warp. Would one of you guys be willing to help us put together a test for Yesod?

#haskell, #yesod, #snapframework on freenode are very helpful.

Re: Web Framework Benchmarks

#157

Post by pmahoney »

xal wrote: post #135

What kind of server did you guys use for your rails test? Thin, Puma, Unicorn? Are you sure you ran it in production environment? Update: Looks like passenger in development mode. Good job you benchmarked a web server that no one uses wile reloading all code between requests. Update2: Ok it seems to run in production mode but still, passenger is not an idiomatic choice.

For the sake of curiosity, I happen to have recently done a benchmark of a "hello world" rack app (literally just responds "hello world" to every request) on a number of Ruby servers (mostly JRuby, but also Puma on MRI).

They were all run in production mode with logging disabled, etc.

http://polycrystal.org/~pat/scratch/microbenchmark.png

Note that a difference from 10k requests per second seems huge compared to 3k, but if you invert it, you get 100 and 333 micro seconds per request, respectively. In a real, non-"hello world" app, these differences are going to be negligible.

Though perhaps it would be more interesting if instead of just responding "hello, world", the app parsed some query parameters or something. But I was mostly interested in the overhead of different JRuby servers, not comparing different app servers (i.e. overhead from Sinatra should be more or less identical whether you're on Puma, Trinidad, or whatever).

Re: Web Framework Benchmarks

#158

Post by goodwink »

voidlogic wrote: post #102
goodwink wrote: post #88

Earlier quoted context omitted.

Wow, directly on tip? That seems to speak very highly of the day-to-day development stability of Go.

A good tip build tends to be more stable then 1.0.3 and has hugely improved performance (most importantly for large application in garbage collection and generation). To select a suitable tip build we use http://build.golang.org/ and https://groups.google.com/forum/?fromgroups#!forum/golang-de... . My recommendation would be to find a one or two week old build that passed all checks, do a quick skim of the mailing li…

Thanks, this comment really helped me in my evaluation of Go today. I had been playing around with 1.0.3 for a couple days, but tip is definitely where it's at.

Re: Web Framework Benchmarks

#159

Post by pfalls »

bilbo0s wrote: post #128
jondot wrote: post #26

Earlier quoted context omitted.

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

I'm not sure many people would use webgo in real life. I don't know... maybe some people... certainly not pros. Also, the 1.0.3 thing is probably dragging on the numbers a bit. 1.1 would boost it a little. Not enough to get it into the top tier... but a little. Also, for Vert.x, they seem to be only running one verticle. Which would never happen in real life. Play could be optimized a bit... but not much. What they h…

for Vert.x, we specified the number of verticals in App.groovy, rather than on the command line, which we think is a valid way to specify it.

Re: Web Framework Benchmarks

#160

Post by vanderZwan »

"Among the many factors to consider when choosing a web development framework, raw performance is easy to objectively measure."

Oh really? Then why did Zed write such an angry rant about how you are doing it wrong?

http://zedshaw.com/essays/programmer_stats.html

Can we please see some standard deviations, at least?

Post reply on HN