Web Framework Benchmarks • Hacker News Board

Re: Web Framework Benchmarks

#351

Post by draegtun » Fri, Mar 29, 2013, 9:59 AM UTC

laumars wrote: post #133

bigpresh wrote: post #120

Earlier quoted context omitted.

No, they're not. They can run via CGI (so, yes, compiled for each request) but that's slow as ass. They run via Plack/PSGI - they can be deployed in a multitude of ways, including via FastCGI or running standalone using built-in webservers, or via Starman - the latter in particular is very fast indeed. With a simple "hello world" type app, > 6000 requests/sec can be easily handled. Obviously an app of more realistic…

Funny enough that SO link was the same one I got my misinformation from as the top answer pointed to CGI benchmarks. Thanks for the correction though, but that still doesn't answer my original question: how does Plack compare with mod_perl?

Plack/PSGI (Perl) == WSGI (Python) == Rack (Ruby)

These are all abstraction layers (for each language) which then can be run seamlessly on top of CGI, SCGI, mod_(perl|python|ruby), etc.

- http://plackperl.org

- http://en.wikipedia.org/wiki/Plack_(software)

- http://en.wikipedia.org/wiki/PSGI

- http://en.wikipedia.org/wiki/Wsgi

- http://en.wikipedia.org/wiki/Rack_(web_server_interface)

Re: Web Framework Benchmarks

#352

Post by commentzorro » Fri, Mar 29, 2013, 10:00 AM UTC

bilbo0s wrote: post #128

jondot wrote: post #26

Earlier quoted context omitted.

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

I'm not sure many people would use webgo in real life. I don't know... maybe some people... certainly not pros. Also, the 1.0.3 thing is probably dragging on the numbers a bit. 1.1 would boost it a little. Not enough to get it into the top tier... but a little. Also, for Vert.x, they seem to be only running one verticle. Which would never happen in real life. Play could be optimized a bit... but not much. What they h…

That first line is truly one of the best comments I've seen when discussing languages. I've clipped it and will use it from now on:

>> I'm not sure many people would use [xxx] in real life. I don't know... maybe some people... certainly not pros.

Re: Web Framework Benchmarks

#353

Post by vixen99 » Fri, Mar 29, 2013, 10:10 AM UTC

tinco wrote: post #289

sergiotapia wrote: post #277

Earlier quoted context omitted.

He's a fantastic software engineer but he is _very_ abrasive. I've read some of his posts claiming he would fight some other developer in person at a conference if he steps up. He would rent out a ring and he would put his yellow belt to practice. Really, I'm not making this up. He sounds like a jerk to work with.

He is actually a very nice person to be around and work with. Besides being professional and experienced he is also outspoken with a firm opinion. You might think that's being a jerk, I think that's honest and reliable.

I am among those appreciative of his contribution to my education. At the same time I wish he'd let the word 'fuck' regain its undoubted impact by not using it on what seems to be every conceivable occasion (excuse the pun).

Re: Web Framework Benchmarks

#354

Post by xd » Fri, Mar 29, 2013, 10:13 AM UTC

TylerE wrote: post #331

tnash wrote: post #137

This is exactly why I decided to use PHP for my startup. I have something along these lines that I hope to blog about in the coming weeks (I tested php-fpm on nginx/go/node.js/silk.js and php won by a landslide when it came to speed). I would love to see php-fpm on nginx included in this test.

The problem with php is that it looks great on (some) micro-benchmarks, but on real apps under real sustained load it certainly turns to cold dog shit from time to time for no apparent reason.

What are you basing this on? I've been using PHP for well over a decade in high load environments and never experienced it turn "to cold dog shit" .. any issues I have experienced had a good reason, not "no apparent reason".

But then, I've never used a PHP framework in all the time I've used it .. maybe that has something to do with me never having had negative issue with PHP.

Re: Web Framework Benchmarks

#355

Post by corford » Fri, Mar 29, 2013, 10:24 AM UTC

bhauer wrote: post #224

knappador wrote: post #112

Utter crap. "Sadly Django provides no connection pooling and in fact closes and re-opens a connection for every request. All the other tests use pooling." But it's free, open-source software and we provide asynchronous database connection pooling for Postgres SQL: https://github.com/iiilx/django-psycopg2-pool

We aim to do Postgres testing soon. As you can imagine, the feedback from this has been awesome. Looking forward to seeing Django on Postgres.

Great to hear you're planning to do some Postgres benchmarks!

To improve fairness, you might want to consider using pgbouncer (setup to only offer simple session pooling) in between the Postgres db and any framework that doesn't have internal support for connection pooling.

E.g. I'd love to see how Flask performs using just the psycopg2 driver (i.e. raw db access, no ORM) and pgbouncer to handle the pooling.

Re: Web Framework Benchmarks

#356

Post by oldpond » Fri, Mar 29, 2013, 10:24 AM UTC

rallison wrote: post #155

goodwink wrote: post #77

Earlier quoted context omitted.

Here's a quick version of the framework optimization index. Higher is better (ratio of framework performance to raw platform performance, multiplied by 100 for scale): Framework Framework Index Gemini 87.88 Vert.x 76.29 Express 68.85 Sinatra-Ruby 67.88 WebGo 51.08 Compojure 45.69 Rails-Ruby 31.75 Wicket 29.33 Rails-Jruby 20.09 Play 18.02 Sinatra-Jruby 15.96 Tapestry 13.57 Spring 13.48 Grails 7.11 Cake 1.17

In the same vein, I was curious to compare the max responses/second on dedicated hardware vs ec2 on a per framework basis. The following is percentage throughput of ec2 vs dedicated (in res/s): cake 18.9% (312 vs 59) compojure 12.1% (108588 vs 13135) django 16.8% (6879 vs 1156) express 16.9% (42867 vs 7258) gemini 12.5% (202727 vs 25264) go 13.3% (100948 vs 13472) grails 7.1% (28995 vs 2045) netty 18% (203970 vs 3671…

You're saying that running a query across the internet to ec2 is 5 times faster than running it on dedicated hardware in the lab? I find that hard to believe.

Re: Web Framework Benchmarks

#357

Post by chrisdone » Fri, Mar 29, 2013, 11:08 AM UTC

pekk wrote: post #153

papsosouid wrote: post #141

Earlier quoted context omitted.

>Given the amount of interest in Haskell I see more people making uninformed "haskell sucks" posts than expressing interest in it. >and Yesod Really? Yesod is the anti-haskell haskell framework.

You and tikhonj are all over the place in here. If you were being downvoted into the gray, you would know. There is no shortage of people praising Haskell every day, this is what is in fashion today. There are also regular posts about Yesod. I conclude that you know perfectly well that Haskell and Yesod are regularly mentioned on HN, but find it inconvenient to have mentioned for some reason I do not fathom.

> There is no shortage of people praising Haskell every day, this is what is in fashion today.

I really don't like this characterization of interest in Haskell. It implies that it's no different from any other language and is just arbitrarily picked up because it's trendy. Learning Haskell is a very substantial investment of time and effort, it is very different from languages that most programmers have used before. It practically tells people “don't even try to like me, I'm high maintenance.”

Re: Web Framework Benchmarks

#358

Post by chrisdone » Fri, Mar 29, 2013, 11:28 AM UTC

jalopy wrote: post #192

As a Rails developer and admirer, this is eye-opening. I love the framework (and Ruby especially), but these numbers bear some serious consideration. 30-50x performance difference gets really... real, no? The standard refrain of "throw more hardware at it" must reconcile with the fact that a factor of 30-50x means real dollars for the same amount of load. Is the developer productivity really that much greater?

It comes as a surprise to me that this comes as a surprise to you. Really, you didn't know Ruby is pretty much as slow as it gets?

Re: Web Framework Benchmarks

#359

Post by oneeyedpigeon » Fri, Mar 29, 2013, 11:34 AM UTC

hnriot wrote: post #147

haha, all those hipster developers using rails can now eat the php guys shorts :) Seriously though, this isn't news to anyone that does this professionally. The further up the abstraction curve you climb, the less performant the code will be. Ease of development vs run-time performance.

Exactly. And since human time is much more expensive than CPU time, very few people are writing their web apps in machine code.

Re: Web Framework Benchmarks

#360

Post by chrisdone » Fri, Mar 29, 2013, 11:40 AM UTC

bradleyland wrote: post #273

jalopy wrote: post #192

As a Rails developer and admirer, this is eye-opening. I love the framework (and Ruby especially), but these numbers bear some serious consideration. 30-50x performance difference gets really... real, no? The standard refrain of "throw more hardware at it" must reconcile with the fact that a factor of 30-50x means real dollars for the same amount of load. Is the developer productivity really that much greater?

Preface: This post is going to come across as a Rails apologist piece, but please read the entire thing before you reach a conclusion. Please also consider that you could apply these same arguments to just about any of the high-level language based frameworks on the list. I use Ruby on Rails in my comparisons, but I'm a huge fan of Node.js, Python/Django, and Go. I fully respect the JVM family of languages as well. I…

Don't conflate Ruby with Rails. Ruby _is_ slow:

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...

and so is Python:

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...

The only time Ruby or Python are fast is when the program is not running Ruby or Python but running some C code underneath. If your programs only consist of that, you can very well say "this ‘Ruby/Python’ code is fast". But as soon as you have something that isn't in your standard library, welcome to the actual language, and welcome to performance problems.

Elaborating on the implications of this: whenever you actually _use_ the language to do some abstraction, you pay heavily for it: http://bos.github.com/reaktor-dev-day-2012/reaktor-talk-slid...