Web Framework Benchmarks

Yes, exactly. I'd love to see some examples of a CRUD based API built on top of h2o. At least this appears to be possible with one of the higher performers (cppcms).

Trust me. You won't hit 424,712 requests per seconds per each node (that is 1,528,963,200 per hour).

You need to have many nodes and a a well distributed and scalable infrastructure to handle 1B req/hr.

There is no point to benchmark your programming-language/framework when doing web development.

When it comes to web, you will hit N different capacity limits before that even with slowest framework, starting from your database, network connections, bandwidth, third-party API rate limits, IO performance etc...

Consider only a few options base on project requirements, ecosystem, productivity, and available human resources. then compare the top two on performance.

You can't compare H2O and Rails on the same list. they are two different animals.

Moreover, I'm sure all of these frameworks are not well optimized for a raw performance benchmark, and also these results are heavily affected by filters you are choosing. Don't spend time on pointless benchmarks.

We use the word "framework" as a term of convenience covering the full spectrum from platforms, micro-frameworks, to full-stack frameworks. We are also liberal with accepting contributions from the community, which means we do indeed include several frameworks that are well outside the mainstream.

That said, in tests such as Fortunes, you will see many full-stack frameworks demonstrating their capability to deliver myriad web application fundamentals (such as request routing, database connection pooling, ORM, XSS countermeasures, character encoding, data structures, and server-side templates) at very high performance levels.

This project establishes a high-water mark of performance. Very few web applications process anything remotely close to the requests per second we're measuring. But using Fortunes as a proxy and applying a coefficient (e.g., 0.005) to adjust for real-world application sizing can give you a very rough but nevertheless potentially useful approximation of real-world expectations. For example, one can guess their application is 200 times more complex than our Fortunes test. Rough back-of-the-envelope math will then estimate a framework from the ultra-high performance tier (~200,000 fortunes/sec) might yield a generous ~1,000 real-world app requests per second on a modern Xeon server. Meanwhile, a framework in the middle tier (~10,000 fortunes/sec) would yield a more constraining ~50 real-world app requests per second. Again, this math is hand-waving and you can interpret the results in whatever way you prefer; you can dismiss the validity of approximating things so coarsely. But my experience is that real-world applications based on frameworks I've used—which are sprinkled among all tiers of our results—do align with this hand-waving approximation.

Edit: I invite you to read the last section of the blog entry about this round [2] where I argue the same point in another way and share one of the tweets from an application developer whose real world application benefited from the performance improvements made to his favorite framework.

[1] https://www.techempower.com/benchmarks/#section=motivation

[2] https://www.techempower.com/blog/2018/06/06/framework-benchm...

Yep. It took me about six months to write a replacement of ActiveRecord with an identical API (for what was needed) that could scale to arbitrarily large number of writes.

Shit, pre-A16Z raise 500px ran on Rails and they were a social network around photography. You think Twitter is bad? Try timelines where you have half a million photographers liking hundreds of photos an hour and every like gets pushed into every feed of the people following them. Guys syncing their entire photo library. It was 5 application servers, 4 MongoDB servers for the timeline with some crazy data structures, one or two MySQL DBs.

I think most web projects these days should be in either Pheonix, Rails, or similar. If you need something really fast here or there just fork off the request in Nginx or compile something in Rust or C and extend it into Ruby (or whatever). Or have a compiled worker that communicates through the DB or Redis. There is this long tail of UI you need to make for every web project and nobody uses it 99.999% of the time. It should be in whatever secure language / framework that brings it to market fastest.

Interesting I just submitted h20 2.3 Beta, but it seems to be not getting much interest on HN. It could have been called 3.0 with so many new features.