Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

Post reply on HN
Print view View original

141–150 of 358 posts

Re: Web Framework Benchmarks Round 4

#141

Post by trailfox »

freework wrote: post #119
trailfox wrote: post #91

Earlier quoted context omitted.

For many systems performance matters. Claiming that there is no difference between Ruby and C is just sticking your head in the sand.

You can build an application in Ruby, deploy it onto 15 machines, and it will outperform the same application written in C and deployed to only one machine. Performance is more of a function of the underlying hardware than the language used to build it.

If a language is 30x less efficient than another language then you would likely need 30x more servers. Many folk are simply not prepared to spend 30x more than they need to on hardware. It's the difference between 20 servers and 600 servers.

Case in point: "How We Went from 30 Servers to 2":

http://blog.iron.io/2013/03/how-we-went-from-30-servers-to-2...

Re: Web Framework Benchmarks Round 4

#142

Post by xaritas »

hippich wrote: post #57

Whenever I hear "Java" I also get association "slow". But looking into this list - java web frameworks doing incredible job!

I think that this lingering perception is an artifact of two things. Firstly, in the mid-to-late 90s, Java was really was slow and back then C/C++ application layers were pretty common (perl if you could get away with it). So initially Java did not compare well, which is why we wasted a couple years on applets. Even as late as 2005 at Amazon, there were many people predicting doom when we introduced the first Java service as a dependency of the home page. Secondly, the early Java web frameworks were highly synchronous, with lots of locking, and there was no evented IO. So sites written that way really were dogs.

Although I hope to never write "public static void main" again (except ironically, of course), and I spend some time dabbling in Python/Ruby/obscure-language land, I'm really happy to see Clojure and Scala doing well here.

Re: Web Framework Benchmarks Round 4

#143

Post by oberhamsi »

cmircea wrote: post #28
showerst wrote: post #25

Earlier quoted context omitted.

Not that PHP is particularly fast, but I notice that Raw PHP does about middle of the pack, but all of the ORM versions seem to do terribly. Interesting stuff. I've usually had pretty good luck with Slim, I'll have to try a version with & without redbean and see how big a difference it makes.

Probably because all those big frameworks have to initialize everything for every request. Parsing, connections, configuration, you name it.

Is there no php environment with a persistent server? Are they really all doing per request startup?

Re: Web Framework Benchmarks Round 4

#144

Post by hilem »

krg wrote: post #8
camwest wrote: post #6

What version of the programming languages are being used?There is a major difference between Ruby 1.9.x and Ruby 2.x

Ruby 2.0.0-p0 http://www.techempower.com/benchmarks/#section=environment

though they're also using Rails 3.2.11, which not only isn't the latest 3.2.* release but doesn't take advantage of Ruby 2.0... it would be more interesting to see the latest Rails 3.2.* on Ruby 1.9.* along with the latest Rails 4 release candidate on Ruby 2.0.0-p0

Re: Web Framework Benchmarks Round 4

#145

Post by oberhamsi »

phillmv wrote: post #118
dllthomas wrote: post #58

Earlier quoted context omitted.

I agree that performance shouldn't dominate the decision, but there's no reason not to be informed by it - it can wind up mattering.

Except benchmarking is really, really hard to get right, and these benchmarks aren't really testing anything that resembles a production app. For all non-trivial apps, by the time you get 100 req/sec your bottleneck is very likely going to be your database.

I think you ran into your own argument: if you are very read heavy (and lots of big sites are), it's all about caching and the DB becomes irrelevant. QED really hard to get right

Re: Web Framework Benchmarks Round 4

#146

Post by rcoder »

EGreg wrote: post #40
coldtea wrote: post #31

It's impressive how well PHP holds up with many queries per request (which is the most common CRUD/webapp scenario). While for no or just one query it's slower than a lot of the other frameworks (due to PHP being slow to parse, startup etc), as soon as we have a lot of DB queries, the C interface to MySQL leaves the other frameworks in the dust. The well known PHP shortcomings aside, that's a nice example of optimizi…

In really scalable sites, you need sharding. Unless your database itself is doing the scaling (such as with Riak), you're going to sometimes hit multiple shards. With PHP and other languages that can't do async, you're going to have to query the DB sequentially, increasing latency proportionally to the number of shards you have to hit. With Node.js and other asynchronous apps, you don't. Disclaimer: mysqli does have…

What you need for sharded queries is concurrency, not necessarily asynchronous requests. Async callbacks (ala Node.js, Twisted Python, Event Machine, etc.) give you a kind of cooperative multitasking, which is one way to have concurrent I/O-bound tasks going; multithreaded programs are another. (Ruby and Python threads are kind of in-between, due to their respective GIL limitations.)

That being said, above a certain scale and complexity level, you probably want the topology of your persistent data store hidden from your web request handlers anyway. For one thing, making requests to N backend shards from M frontend web workers starts to get bad when N and M are both large; for another, introducing really complex scatter-gather query logic into your request-handling pipeline can be a maintenance and debugging nightmare.

Introducing a proxy or data-abstraction service in between cuts down on the number of open connections and lets you change the data storage topology without updating frontend code.

Re: Web Framework Benchmarks Round 4

#148

Post by Periodic »

First off, I love the work you're doing, keep it up.

Benchmarks like this are designed to be the starting-point of a discussion an investigation, and not as anything meaningful in their own right. Boiling it down a framework to one performance number ignores the many, many nuances of a framework.

What surprises me most is the difference between different frameworks. A few years ago the mantra seemed to be "Use Rails, Django or a similar full-stack framework. Speed of deployment trumps everything!" Over the last few years I've seen a shift as people are trying to get more performance from limited hardware. Personally I'm intrigued by how a fairly innocent decision early in the project (of what language/framework) may have profound performance implications in the long run.

For myself, I've been looking for a good functional-programming framework. Just looking at this gives me a good list of frameworks to start looking at. It feels to me that a framework that performs well is likely well engineered, so the ones that perform better will go at the front of my queue for investigations.

Re: Web Framework Benchmarks Round 4

#149

Post by rallison »

phillmv wrote: post #118
dllthomas wrote: post #58

Earlier quoted context omitted.

I agree that performance shouldn't dominate the decision, but there's no reason not to be informed by it - it can wind up mattering.

Except benchmarking is really, really hard to get right, and these benchmarks aren't really testing anything that resembles a production app. For all non-trivial apps, by the time you get 100 req/sec your bottleneck is very likely going to be your database.

The general point of these benchmarks is not to resemble a full production app, but to provide a baseline measurement. From the original blog post[1]:

This exercise aims to provide a "baseline" for performance across the variety of frameworks. By baseline we mean the starting point, from which any real-world application's performance can only get worse. We aim to know the upper bound being set on an application's performance per unit of hardware by each platform and framework.

But we also want to exercise some of the frameworks' components such as its JSON serializer and data-store/database mapping. While each test boils down to a measurement of the number of requests per second that can be processed by a single server, we are exercising a sample of the components provided by modern frameworks, so we believe it's a reasonable starting point.

So, yes, these benchmarks should not be the only factor in choosing a framework, but they do provide a possibly important data point (depending on the specific scenario).

[1] http://www.techempower.com/blog/2013/03/28/frameworks-round-...

Re: Web Framework Benchmarks Round 4

#150

Post by mrtksn »

gtaylor wrote: post #63
reactor wrote: post #36

Off all the top performers, Go seems to be the only sane choice to write a web app. Moreover it is at the sweet spot; expressive, flexible, simple, super performant, good community etc. I think it is convincing enough for me to give Go a serious look for our new app.

As someone who is also really enjoying Go, I think you need to add a huge, gigantic disclaimer before making a statement like this: Go's ecosystem of web development packages is in its infancy. You're not going to find any super-well-documented, super mature/stable web frameworks (thought a few are showing great promise). and some of the individual components (for example, Gorilla) are looking very good, but still ha…

how easy is to use GO with other frameworks, let's say PHP? Would it be possible to write an application that uses PHP for some tasks, so you can benefit from the speed of GO and the the maturity of PHP?
Post reply on HN