Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks

techempower.com

Post reply on HN
Print view View original

11–20 of 116 posts

Re: Web Framework Benchmarks

#11

Post by nobleach »

brightball wrote: post #10
wheaties wrote: post #3

I love these but I've also seen in some of the implementations that I care about a bit of benchmark gaming. That saddens me and I wonder just how much of it is happening in languages and frameworks that I don't follow.

It's interesting to look at the source code. There's definitely some heavy differences in how each implementation is handling underlying aspects of the tests. Just looking at the database related ones to see about some of the differences in interesting. It also doesn't look like the Elixir code has really been updated in about 2 years (aside from version bumps). It's still using a JSON encoder (Poison) that's 4x slow…

That's the beauty of these "pinewood derbies" though. Sometimes they sit dormant until some kind soul says, "HEY!, that's not representative of my favorite framework!" and submits a PR to showcase its true capabilities.

Re: Web Framework Benchmarks

#12

Post by christophilus »

bilbo0s wrote: post #7

I had not checked these for a good long while. (Probably around round 10 or 11.) Looking back in on it now, holy MAN! C (C++) and Java. Wow. Even though I'm old, I'd sort of forgotten how speedy they can be. It's strange, because I actually work with Java and C in making streaming game engines. And you kind of get used to thinking in terms of millions on a set of hardware. So you think Java and C are slow. Then you r…

> Anything other than C or Java really

Rust, C#, and Go also seem to fit the bill, according to these benchmarks. C# particularly has rocketed up since the last time I looked in depth at these. Pretty impressive work from the .NET Core folks, I'd say.

Re: Web Framework Benchmarks

#13

Post by akmittal »

bilbo0s wrote: post #7

I had not checked these for a good long while. (Probably around round 10 or 11.) Looking back in on it now, holy MAN! C (C++) and Java. Wow. Even though I'm old, I'd sort of forgotten how speedy they can be. It's strange, because I actually work with Java and C in making streaming game engines. And you kind of get used to thinking in terms of millions on a set of hardware. So you think Java and C are slow. Then you r…

I'm not sure how many companies use C, C++ for Web development. In my opinion C/C++ wont be good idea to use for most. It is about choosing the best ration b/w tool speed and developer speed(productivity).

There are very few companies which requires million per second request. For me Java, C#, Go, Node provide the best compromise b/w speed and productivity.

Re: Web Framework Benchmarks

#14

Post by christophilus »

argonium wrote: post #8

I've been following these benchmarks for some time, and am always shocked that Spring does so poorly (it's 7% here). I haven't had any performance issues with Spring in production, so these benchmarks are puzzling. Are the other frameworks really that much faster in practice?

What kind of traffic do you see in production? Rails gets hammered in these benchmarks, but does just fine for most companies' requirements. It's pretty rare that you'd actually need to eke out the raw numbers that the top contenders get. I'm of the opinion that-- within reason-- developer ergonomics, and ability to quickly solve business problems are more important than raw performance, so long as performance is good enough by some agreed upon metric.

Re: Web Framework Benchmarks

#15

Post by onion2k »

These are fun benchmarks, but every single framework in the top 250 or so is more than fast enough for most apps. Even the slowest in the list manages 366/requests per second, which is probably enough to prove an idea before optimizing for speed.

We're approaching the point where speed is essentially a solved problem unless you're at Google-scale.

Re: Web Framework Benchmarks

#16

Post by sbhn »

christophilus wrote: post #12
bilbo0s wrote: post #7

I had not checked these for a good long while. (Probably around round 10 or 11.) Looking back in on it now, holy MAN! C (C++) and Java. Wow. Even though I'm old, I'd sort of forgotten how speedy they can be. It's strange, because I actually work with Java and C in making streaming game engines. And you kind of get used to thinking in terms of millions on a set of hardware. So you think Java and C are slow. Then you r…

> Anything other than C or Java really Rust, C#, and Go also seem to fit the bill, according to these benchmarks. C# particularly has rocketed up since the last time I looked in depth at these. Pretty impressive work from the .NET Core folks, I'd say.

Plus one for c#, with compiler optimisations I was able to generate millions of sha256s a second, many more than my c++ version of the same code. Then I discovered CUDA, then I got bored and went onto something else.

Re: Web Framework Benchmarks

#17

Post by tigershark »

akmittal wrote: post #13
bilbo0s wrote: post #7

I had not checked these for a good long while. (Probably around round 10 or 11.) Looking back in on it now, holy MAN! C (C++) and Java. Wow. Even though I'm old, I'd sort of forgotten how speedy they can be. It's strange, because I actually work with Java and C in making streaming game engines. And you kind of get used to thinking in terms of millions on a set of hardware. So you think Java and C are slow. Then you r…

I'm not sure how many companies use C, C++ for Web development. In my opinion C/C++ wont be good idea to use for most. It is about choosing the best ration b/w tool speed and developer speed(productivity). There are very few companies which requires million per second request. For me Java, C#, Go, Node provide the best compromise b/w speed and productivity.

Vertx in Java is basically the fastest across all the benchmarks, putting it in the same speed/productivity bucket as node doesn’t make much sense...

Re: Web Framework Benchmarks

#18

Post by argonium »

christophilus wrote: post #14
argonium wrote: post #8

I've been following these benchmarks for some time, and am always shocked that Spring does so poorly (it's 7% here). I haven't had any performance issues with Spring in production, so these benchmarks are puzzling. Are the other frameworks really that much faster in practice?

What kind of traffic do you see in production? Rails gets hammered in these benchmarks, but does just fine for most companies' requirements. It's pretty rare that you'd actually need to eke out the raw numbers that the top contenders get. I'm of the opinion that-- within reason-- developer ergonomics, and ability to quickly solve business problems are more important than raw performance, so long as performance is goo…

Good point. Thanks for the feedback. Our sites don't get a huge amount of traffic, so it's possible Spring doesn't have as good a concurrency story (or it's due to memory usage) as the higher-ranked frameworks, so it's been sufficient for our needs.

Re: Web Framework Benchmarks

#19

Post by bhauer »

kizer wrote: post #9

Is 'Fortunes' a test involving the quote generating program? Also, is the difference between 'Single Query' and 'Multiple Queries' concurrency? I always look at these but I have trouble understanding the test cases.

The best resource to answer your questions is the Requirements for the various test types [1]. But in brief:

* Yes, Fortunes is named after the Unix tool of the same name. In our case, it's a test that executes a query of all rows in a table, adds an additional item, sorts the values in the application code, escapes the values as a XSS countermeasure, and then renders them using a server-side templating library.

* Yes, the single-query test is always executing a single-query per HTTP request and is measured at various concurrency levels. The multi-query test is measured with consistent concurrency and varies the number of queries executed per HTTP request.

[1] https://www.techempower.com/benchmarks/#section=code

Re: Web Framework Benchmarks

#20

Post by Spartan-S63 »

christophilus wrote: post #12
bilbo0s wrote: post #7

I had not checked these for a good long while. (Probably around round 10 or 11.) Looking back in on it now, holy MAN! C (C++) and Java. Wow. Even though I'm old, I'd sort of forgotten how speedy they can be. It's strange, because I actually work with Java and C in making streaming game engines. And you kind of get used to thinking in terms of millions on a set of hardware. So you think Java and C are slow. Then you r…

> Anything other than C or Java really Rust, C#, and Go also seem to fit the bill, according to these benchmarks. C# particularly has rocketed up since the last time I looked in depth at these. Pretty impressive work from the .NET Core folks, I'd say.

For Rust, I'm interested to see what the future holds once the community coalesces around an async IO interface. Additionally, when procedural macros stabilizes (for real), the possibilities for ergonomic frameworks in Rust become endless.

Rocket has been really interesting to work with, but it only works on nightly since it relies on experimental codegen/procedural macro features. Once procedural macros stabilizes, all the codegen can be ported to that. It doesn't support async IO yet, but that's because it's waiting for the community to coalesce before diving into it.

I think there's a future where Rust web frameworks are as expressive as Rails or Laravel but bring type safety and sound, zero cost abstractions to the table.

Post reply on HN