Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

161–170 of 358 posts

Re: Web Framework Benchmarks Round 4

#161
I implemented the Ringo app for this benchmark and of course ran it against Node and a couple of others to see how we would perform in this neighborhood before I opened the pull request.

And since that day I've been wondering: why does NodeJs (=V8 JS engine in C) talking to MongoDB have higher response times and latency than Ringo (=Rhino JS engine on JVM) talking to MySQL. The only thing where Node beats us JVM guys seems to be the JSON response test.

Re: Web Framework Benchmarks Round 4

#162
post #57

Whenever I hear "Java" I also get association "slow". But looking into this list - java web frameworks doing incredible job!

I think that this lingering perception is an artifact of two things. Firstly, in the mid-to-late 90s, Java was really was slow and back then C/C++ application layers were pretty common (perl if you could get away with it). So initially Java did not compare well, which is why we wasted a couple years on applets. Even as late as 2005 at Amazon, there were many people predicting doom when we introduced the first Java se…

In my opinion, you hit the nail on the head with regard to a view on Java (being that it is "slow"). For many years in the early going, it was slow, but it has come around so much.

That being said, as a day-to-day Java web-developer, I cannot honestly remember the last time I wrote "public static void main".

Re: Web Framework Benchmarks Round 4

#163
post #63
post #36

Off all the top performers, Go seems to be the only sane choice to write a web app. Moreover it is at the sweet spot; expressive, flexible, simple, super performant, good community etc. I think it is convincing enough for me to give Go a serious look for our new app.

As someone who is also really enjoying Go, I think you need to add a huge, gigantic disclaimer before making a statement like this: Go's ecosystem of web development packages is in its infancy. You're not going to find any super-well-documented, super mature/stable web frameworks (thought a few are showing great promise). and some of the individual components (for example, Gorilla) are looking very good, but still ha…

While we're on the subject... Has anyone tried out Revel[.]? If so, what are your thoughts on it?

[.] https://github.com/robfig/revel

Re: Web Framework Benchmarks Round 4

#164
post #40
post #31

It's impressive how well PHP holds up with many queries per request (which is the most common CRUD/webapp scenario). While for no or just one query it's slower than a lot of the other frameworks (due to PHP being slow to parse, startup etc), as soon as we have a lot of DB queries, the C interface to MySQL leaves the other frameworks in the dust. The well known PHP shortcomings aside, that's a nice example of optimizi…

In really scalable sites, you need sharding. Unless your database itself is doing the scaling (such as with Riak), you're going to sometimes hit multiple shards. With PHP and other languages that can't do async, you're going to have to query the DB sequentially, increasing latency proportionally to the number of shards you have to hit. With Node.js and other asynchronous apps, you don't. Disclaimer: mysqli does have…

Some of the fastest implementations you see in these tests are not asynchronous.

With Servlet for example, a worker thread is chosen from Resin's thread pool and used to handle a request. The Servlet then executes 20 queries sequentially and returns the resulting list data structure. This is Servlet 3.0 but not using Servlet 3.0 async.

Async isn't making the top performers fast. Being fast is making them fast.

Re: Web Framework Benchmarks Round 4

#165
post #59

Earlier quoted context omitted.

Well no matter what you use, you should always benchmark for that exact reason. Making things go fast takes a lot of performance engineering, and the JVM has millions of dollars worth of tuning put into it. BTW, if you like Node.js, you should probably look at Vert.x. I haven't used it, but it's a similar concept, it runs on the JVM, and it seems to spank Node.js.

When you benchmark Vert.x on a multicore system vs 1 node process. Now add multiple node processes to occupy the cores and store changes completely

Indeed. Node's IPC costs are orders of magnitude slower than Java's volatile or atomic datatypes. I would only choose Node for stateless or trivially parallelizable problems--e.g., those where I could push the state problem into a runtime with real threads.

http://aphyr.com/posts/244-context-switches-and-serializatio...

Re: Web Framework Benchmarks Round 4

#166
post #137

While I really enjoy results, I still would prefer to see smaller number of frameworks with specific tasks that are common in web development, logging in (assume user and pass are provided), listing of something. These two would be way more meaningful then anything else to me. Again, thank you very much for hard work. I think for some there are some revelations there, like that C framework.

We aim to add more tests in time. Take a look at the new Fortunes test. That test lists rows from a database, sorts them, and then renders the list using a server-side template. It's only implemented in 17 of the 57 frameworks right now, but we hope to have better coverage on that in time as well.

Yes something along those lines was what I was thinking. Thank you.

Re: Web Framework Benchmarks Round 4

#169
Before looking at the benchmark results, I took a glance at the Node source and I expected it to perform worse than it did previously. It does almost universally. Not only haven't the glaring perf issues remained since round 1, it's added more. In the real world, when you look at a metric that says your req/s is a bottleneck, which is what this benchmark is loosely simulating, you'd fix it. You wouldn't just say "nope, that's what this framework does, sorry boss."

I still don't find these benchmarks very useful. From the looks of the comments, a lot of you don't really either (even if you don't realize it).

For example, a lot of people in these comments want to correlate language speed with performance in these benchmarks, by arguing specific examples, but comparing almost any two frameworks/platforms in this "benchmark" is an apples to non-apples comparison, and the result is actually full of counter examples (faster languages performing more poorly). That should instantly tell you that this benchmark isn't telling you what you think it's telling you, and that you haven't really derived any value from it.

Perhaps the biggest reason I don't find value here is that every product here does wildly different things. It's like comparing wrenches to hammers to screwdrivers to 3D printers.

I also want to point out to people who say that this is a "comparison" of frameworks that it is emphatically not a comparison. What is the value of a framework? Is it speed? Atypically. And this "benchmark" tends to point at such cases as "being better" because they do better in this specific task. A framework/platform's value lies in features and abstractions. This does not compare those.

I will gladly build a "framework" in NodeJS that is only capable of doing the tasks in this benchmark as fast and with as little overhead as possible. You would NEVER use it in the real world, but it would be a beast at serializing JSON and making repeated database queries in an insecure fashion. But score here is the important factor, right?

Re: Web Framework Benchmarks Round 4

#170

First off, I love the work you're doing, keep it up. Benchmarks like this are designed to be the starting-point of a discussion an investigation, and not as anything meaningful in their own right. Boiling it down a framework to one performance number ignores the many, many nuances of a framework. What surprises me most is the difference between different frameworks. A few years ago the mantra seemed to be "Use Rails,…

>Over the last few years I've seen a shift as people are trying to get more performance from limited hardware.

Part of that shift is also that other frameworks have learned and integrated a lot from Rails/Django. The productivity/time-to-launch gap isn't as significant as it used to be, so other factors like performance, compatibility with pre-existing infrastructure (eg for JVM-based frameworks), security, etc. are gaining more influence in the decision about what to use.

Post reply on HN