Live data from Hacker News

Frameworks Round 6

techempower.com

21–30 of 135 posts

Re: Frameworks Round 6

#21
post #13

Does anyone have more info on "cpoll-cppsp"? Hearing about it for the first time, it seems to be smoking all the other frameworks.

I was actually just checking it out for myself.

http://sourceforge.net/projects/cpollcppsp/

https://github.com/xaxaxa/workspace/

Seems to be a C++ framework with dynamics in the form of C++ code using an ASP style for embedding into the resource.

Re: Frameworks Round 6

#22
post #6

Earlier quoted context omitted.

I guess you need to change the Hardware to "Win" to see it shining.

Indeed, mono performance isn't great. That is the problem with C#, you need ultimately to run it on IIS for web platform deployment, sadly under anything else it is rather slow. Also these tests are a bit unfair, as I think they give benefit in some areas to frameworks that cache by default, rather than by request. Ultimately, what I want to know is total cost, that means hardware, licenses / support agreements, deve…

I missed your later reply in that other thread. If the node.js implementation is caching results from the database, it should be modified to not do so. The original implementation did not, but it has seen several updates from node.js SMEs. Can you point to what makes you suspect that it is caching? The requirements do not allow for caching [1]. Future test types (perhaps in Round 7) will allow for caching, but none of the present test types allow for it.

[1] http://tewebdev.techempower.com/benchmarks/#section=code

Re: Frameworks Round 6

#25
post #24

Honest question: Why are most of the frameworks that I know of in bottom half? When does speed becomes relevant?

Three reasons as to the hierarchy:

1) Most popular frameworks you would encounter run on higher level dynamic languages such as PHP, Ruby or Python. These languages are a decent bit slower than performance optimized languages. The languages towards the top are all C, Java, Go, etc.

2) The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.

3) In general, it's more likely you will have heard of older frameworks than newer ones. The frameworks towards the top are generally all less than 2 years old (excluding Java servlets which have just always been decent). This means they make use of newer designs such as non-locking and asynchronous connection handling.

When does speed become relevant? It's actually always relevant as soon as your product gets traction. For example, 1 server at the top of that chart may be able to handle the same load as 10 servers towards the bottom. If you're using a framework at the bottom and paying heavily for 10 servers, you could increase your startups runway substantially by swapping. However, swapping may cost more in engineer-hours than the servers.

Basically, the relevance of speed is directly related to your margin. If your margin is high (investment banking maybe?), then speed is almost irrelevant, you can afford a whole building of servers. If you're competing heavily on mass market web apps, speed is fairly critical as you can out price competition.

Re: Frameworks Round 6

#26

For such a young language, I'm really surprised by the performance of Go there. Quite awesome.

Round 4 of these tests is what triggered my dive into Go this past month, and it has been a revelation. If you add in the concurrency capabilities, the language simplicity, and nice balance between overly terse syntax (Python one liners...) and overly verbose typed languages (Java), the whole package is even more impressive.

The only real thing I think it is missing is a "high productivity" application framework like Rails to elevate it from "damn, I'm impressed" to "it would be ill advised to use almost anything else".

Re: Frameworks Round 6

#28
post #25
post #24

Honest question: Why are most of the frameworks that I know of in bottom half? When does speed becomes relevant?

Three reasons as to the hierarchy: 1) Most popular frameworks you would encounter run on higher level dynamic languages such as PHP, Ruby or Python. These languages are a decent bit slower than performance optimized languages. The languages towards the top are all C, Java, Go, etc. 2) The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle…

Great response, RyanZAG.

To add to this, I'd encourage you to look at the source code used to implement the tests in the higher performance frameworks. Ryan's third point is very relevant: you may find that modern frameworks leverage both high-performance platforms and much of the pragmatic thinking that was introduced a few years back in the frameworks with which you are more familiar.

Source code link: https://github.com/TechEmpower/FrameworkBenchmarks/

Re: Frameworks Round 6

#29
Finally! I was disappointed when this round was postponed.

I couldn't wait to see how well my framework (Jester; https://github.com/dom96/jester) performed, not only because it was my framework but also because it is written in Nimrod. I am a bit disappointed by the results but at least there is a lot of room to improve now, I didn't have much time to properly implement concurrency for Jester so I opted for just spawning a couple of processes which is definitely not ideal.

I hope that Round 7 will go better for Jester. In the meantime I hope that you will check out the Nimrod programming language (http://nimrod-code.org) despite the results that my framework achieved.

To the team that carried out these benchmarks, thank you!

Re: Frameworks Round 6

#30
post #29

Finally! I was disappointed when this round was postponed. I couldn't wait to see how well my framework (Jester; https://github.com/dom96/jester ) performed, not only because it was my framework but also because it is written in Nimrod. I am a bit disappointed by the results but at least there is a lot of room to improve now, I didn't have much time to properly implement concurrency for Jester so I opted for just spa…

Finally indeed! In future rounds, I hope that we can spend less time trying to resolve problems--if there are problems, we'll just post the results and move on to the next round. Better to keep iterating, and allow the experts who work with each framework resolve issues as they come up.

Thanks very much for contributing Jester to the test! I really appreciate your spirit and hope to see it improve in Round 7 as well. I'm looking forward to seeing the next pull request. :)

Post reply on HN