Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Techempower Web Framework Benchmarks Round 10

techempower.com

Post reply on HN
Print view View original

31–40 of 66 posts

Re: Techempower Web Framework Benchmarks Round 10

#31

Post by EugeneOZ »

wheaties wrote: post #19
Cyph0n wrote: post #10

Earlier quoted context omitted.

That is quite surprising. Anyone have an idea why that is?

That's easy to answer, instead of using a relational database or even PostgreSQL's JSON data store, it's using MongoDB. I hate MongoDB but for tests like these where there's absolutely no writes what so ever, Mongo is going to fly. Hence, I read this as comparing MongoDB read performance to PostgreSQL, MySQL or SQL Server perf (or any of the other DB that are listed.) That's why benchmarks like these have to be scrut…

If each framework use different DB, then all results with DB queries are useless (for me). Very frustrating. Really, any framework with Redis will have HUGE allowance

Re: Techempower Web Framework Benchmarks Round 10

#32

Post by wmf »

redstripe wrote: post #20

There's something I don't understand in these comments. Why is everyone interested in language comparisons instead of the huge difference between EC2 and bare metal?

Because people think they can't use bare metal and thus its advantages are irrelevant.

Re: Techempower Web Framework Benchmarks Round 10

#33

Post by stefantalpalaru »

redstripe wrote: post #20

There's something I don't understand in these comments. Why is everyone interested in language comparisons instead of the huge difference between EC2 and bare metal?

From what I know about the Xen virtualization used by EC2 the instance variability is too high to get consistent results over time and the overhead for this type of benchmarks is so big it's not even funny.

That's why the bare metal results are the only relevant ones. The playing field is fair and stable. Let the battle begin ;-)

Re: Techempower Web Framework Benchmarks Round 10

#34

Post by emn13 »

kainsavage wrote: post #13
Cyph0n wrote: post #10

Earlier quoted context omitted.

That is quite surprising. Anyone have an idea why that is?

Actually, not really. We checked the code to ensure that there was no gaming the system and it definitely APPEARS to be making separate database queries as we require in our rules. In fact, we had this same question in round 9 and had a number of people audit it. We cannot explain it other than it might be pretty darn fast.

A better requirement would be to define some minimum level of durability, and some minimal level of freshness in the face of concurrent modifications.

Frankly, who cares if a caching driver avoids some database queries entirely if it still provides the same level of durability and freshness guarantees? If mongo+redis are OK, what's wrong with a plain hashtable?

Re: Techempower Web Framework Benchmarks Round 10

#36

Post by Joeri »

Every time I'm struck by how slow the big PHP frameworks are compared to raw PHP. Either nobody cares about making those tests perform better, or something is very wrong in the architecture of those frameworks.

I expect the cause is that too much code is being loaded for every request. PHP tears down and rebuilds the world for every request, and the popular frameworks load a lot of code and instantiate a lot of objects for every request.

Re: Techempower Web Framework Benchmarks Round 10

#37

Post by perishabledave »

bhauer wrote: post #4
tolas wrote: post #3

Still no Elixir/Phoenix inclusion? I'd really love to see how it stacked up.

Coming in Round 11 [1]! We'll aim to have Round 11 out much quicker than 10. [1] https://github.com/TechEmpower/FrameworkBenchmarks/pull/1510

Thanks! I'd like to see this as well.

Re: Techempower Web Framework Benchmarks Round 10

#39

Post by plorkyeran »

EugeneOZ wrote: post #31
wheaties wrote: post #19

Earlier quoted context omitted.

That's easy to answer, instead of using a relational database or even PostgreSQL's JSON data store, it's using MongoDB. I hate MongoDB but for tests like these where there's absolutely no writes what so ever, Mongo is going to fly. Hence, I read this as comparing MongoDB read performance to PostgreSQL, MySQL or SQL Server perf (or any of the other DB that are listed.) That's why benchmarks like these have to be scrut…

If each framework use different DB, then all results with DB queries are useless (for me). Very frustrating. Really, any framework with Redis will have HUGE allowance

You can filter by DB, and most of the frameworks (including the Dart ones) are run against multiple DBs.

Re: Techempower Web Framework Benchmarks Round 10

#40

Post by buster »

redstripe wrote: post #20

There's something I don't understand in these comments. Why is everyone interested in language comparisons instead of the huge difference between EC2 and bare metal?

Indeed.. I am wondering the same.. I fear that at some point the baseline for performance will be some virtualized server and people will forget just how fast real hardware can be.. At some point people will think it's normal to host a simple blog on 5 virtual servers ;)

Also, by its nature EC2 should be terrible for serious benchmarking, since you have no control whatsoever about the infrastructure.

Post reply on HN