Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

Post reply on HN
Print view View original

111–120 of 358 posts

Re: Web Framework Benchmarks Round 4

#111

Post by notJim »

cmircea wrote: post #28
showerst wrote: post #25

Earlier quoted context omitted.

Not that PHP is particularly fast, but I notice that Raw PHP does about middle of the pack, but all of the ORM versions seem to do terribly. Interesting stuff. I've usually had pretty good luck with Slim, I'll have to try a version with & without redbean and see how big a difference it makes.

Probably because all those big frameworks have to initialize everything for every request. Parsing, connections, configuration, you name it.

Raw php also has to initialize connections, and it does really well, so I don't think that's it. To me, that points to overhead from initializing the objects as the bottleneck. I would have thought that with APC, this wouldn't be a major issue, though, so I wonder if that's still not it.

Re: Web Framework Benchmarks Round 4

#112

Post by phillmv »

hippich wrote: post #57

Whenever I hear "Java" I also get association "slow". But looking into this list - java web frameworks doing incredible job!

Because it is slow. Compared to C/C++/langs without runtimes [albeit with JIT compilation, for long running applications this is becoming a non issue].

Compared to the current crop of dynamic language interpreters, waaaay more engineering time and talent has been poured into optimizing the jvm.

People I think forget that Java was a more user friendly C++; the price you paid was somewhat slower apps, but that's OK because you write more robust apps more easily. Rinse and repeat for Ruby/Python/Your Lang Here.

Re: Web Framework Benchmarks Round 4

#113

Post by bhauer »

saosebastiao wrote: post #15

I love what you guys are doing. This is by far the most comprehensive (in terms of number of frameworks) comparison of web frameworks. I also am a big fan of the new filtering metadata. However, I'm starting to think that all of the advocates of various frameworks are now conspiring independently to make this comparison meaningless...any framework (except Cake for some reason) can be superoptimized towards a small se…

Hi saosebastiao,

As Pat points out, we definitely look forward to implementing some more computationally-intense request types in the future. This round does include the first server-side template test. We'd like to hear the community's opinions about more tests.

That said, I feel most of the frameworks' implementations of the existing tests are not cheating. Our objective in this project is to measure every framework with realistic production-style implementation of the tests. No doubt there is temptation to trim out unnecessary functionality and focus on the benchmark's particular behavior. We have attempted to identify any such tests that remove framework features to target the benchmark as "Stripped" and those can now be filtered out from the list.

In other words, our aim is that the implementation of each framework's test is idiomatic to that framework and platform. And if that's not the case for a test, we want to correct it.

Your concern could be clarified by pointing out that framework authors may be tuning up their JSON serialization, database connection pools, and template processing in order to improve their position on these charts. And, to be clear, I have already seen evidence of that in my interaction with framework authors. To that concern, however, I would say: That is awesome. I want those features to be fast.

Re: Web Framework Benchmarks Round 4

#114

Post by krg »

ihsw wrote: post #95

It's very depressing to see Symfony2 at the bottom of these lists. Although there are quite a few performance optimizations that can be done to improve this, there are few excuses for such poor performance by default.

If you have improvements to the test (that would be realistic for a production deployment) then please submit a pull request. We definitely want to show each framework doing the best it can do.

A few frameworks have a "stripped" version (just Django and Rails so far) to try to show the best that can be achieved when typical functionality is stripped out. Essentially optimizing for this test, which is interesting even if it isn't the point of these benchmarks. If you think Symfony2 would benefit from a separate "stripped" test then please consider a submitting a pull request with that.

Re: Web Framework Benchmarks Round 4

#115

Post by swah »

papsosouid wrote: post #70
swah wrote: post #14

Earlier quoted context omitted.

Agreed, but the language is sufficiently (edit) MORE verbose than Python that it is still a valid strategy to write in Python first, and then rewrite in Go for performance..

If go is less verbose, then why write it in a more verbose language first? The reality is, go and python are pretty much even for verbosity, and there is absolutely no benefit to writing your app in python first and then go. You'd be better off writing it in go and then rewriting it in go if you want the benefits of learning from your first attempt.

I mean't more verbose, sorry.

Re: Web Framework Benchmarks Round 4

#117

Post by swah »

artursapek wrote: post #60
swah wrote: post #14

Earlier quoted context omitted.

Agreed, but the language is sufficiently (edit) MORE verbose than Python that it is still a valid strategy to write in Python first, and then rewrite in Go for performance..

Wow. Great to hear. My current big project is all in Python. Maybe a rewrite is in my future if I find the project successful.

I mean't more verbose, sorry.

Re: Web Framework Benchmarks Round 4

#118

Post by phillmv »

dllthomas wrote: post #58
freework wrote: post #49

I really don't like these benchmarks. Its like benchmarking Fizzbuzz or something. Frameworks don't do anything. No one chooses a framework (at least I don't) based on performance. You choose one framework over the other because you like the API and/or language. I myself am a framework author (giotto, a python framework that was not included in these benchmarks). If my framework had been included, I'm sure it would e…

I agree that performance shouldn't dominate the decision, but there's no reason not to be informed by it - it can wind up mattering.

Except benchmarking is really, really hard to get right, and these benchmarks aren't really testing anything that resembles a production app.

For all non-trivial apps, by the time you get 100 req/sec your bottleneck is very likely going to be your database.

Re: Web Framework Benchmarks Round 4

#119

Post by freework »

trailfox wrote: post #91
lucian1900 wrote: post #69

Earlier quoted context omitted.

Not sure why this is downvoted so badly, the comment is largely correct. This benchmark is even less useful than alioth's shootout, I'm not sure why there is so much effort put into it :)

For many systems performance matters. Claiming that there is no difference between Ruby and C is just sticking your head in the sand.

You can build an application in Ruby, deploy it onto 15 machines, and it will outperform the same application written in C and deployed to only one machine. Performance is more of a function of the underlying hardware than the language used to build it.

Re: Web Framework Benchmarks Round 4

#120

Post by seldo »

Feature request: it would be nice to have permalinks (even if they were really messy URLs) to filter-sets so that I can share the slice of the benchmarks I'm looking at with people, without having to list the filters manually.
Post reply on HN