Live data from Hacker News

Frameworks Round 6

techempower.com

41–50 of 135 posts

Re: Frameworks Round 6

#41
An honest confession from someone inspired by these comparisons. I've been following these benchmarks very closely right from round one and ever since I've been waiting to see my favorite framework, Ruby on Rails perform decently to some extent. I waited till the last round to see some improvements and while I DID see some marginal improvements, it wasn't as expected.

This then provoked me to do some basic math. Imagine a real world startup scenario wherein you need to your application to deliver API responses (JSON) or even ordinary HTTP responses. If you use a framework from the bottom of the list (like Rails), then you may be able to serve, say, 'X' customers on your site using your application at a time. Now, if you use something from the top of the list, you may be able to serve roughly about 20 times (20X) more number of customers on a single server. This not only means less cost, it also means you can worry about scaling and stuff much much later than you would have to, unlike on a low performant framework.

For this reason, I decided to conduct an experiment for myself. Pick up a framework from the top of the benchmarks and experiment with the learning curve and the language. Well, after carefully evaluating, I decided JAVA wasn't for me (too much of a beast). But Scala seemed to hit the sweet spot with excellent language (1 line of Scala = 10 lines of Java on an average) features along with the trust of the JVM.[1]

So, I chose Scala and started experimenting with it. I started with the play framework and was not fond of certain decisions that were made within the framework. Also, the play framework is nothing like Ruby on Rails (but it claims to be so) and was more of self-serve and also quite heavy for my taste. Hence, I went even farther up the list and chose Scalatra. Generally, framework selection is a matter of personal preference and each one should choose whatever aligns with their philosophy. For me, it was scalatra.

Frankly, I haven't been able to ship out features at the speed at which I used to, with Rails, but there was something very important that I learnt too - Don't get comfortable with full stack frameworks without understanding what goes underneath. Rails suddenly seemed like an enemy to me because it was doing everything so magically that I literally had NO IDEA until I had to implement them myself with Scalatra. For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch???? I knew none of these, until I had to implement them myself for my application on top of Scalatra. It was seriously a pain because for the first one week, I did absolutely nothing, in terms of progress with features. But later, I started loving this way of development rather than relying on 'magic' frameworks. Now 'magic' seems scary for me because I cannot actually see what's happening underneath.

So, to cut a long story short, Should you choose the fastest framework? Or should you follow the 'ship it first' policy with an average framework??

My advice - Start with a full stack framework, something like Rails or Django, but also in parallel, try to understand how every bit of your framework works and try to implement it into a micro framework based on something from the top of the benchmarks (like Go, Scala, etc.). Most importantly LEARN. Something new everyday! And slowly shift your development towards these high performant frameworks as and when you see fit.

This is just based on my experience :)

[1] 1 JVM = 10 Thins, Source: http://www.slideshare.net/pcalcado/from-a-monolithic-ruby-on...

Re: Frameworks Round 6

#42
post #2

Thanks for posting this, amarsahinovic. This is the latest round of our ongoing project to benchmark the performance of fundamental services provided by modern web application frameworks. With the continued contributions of the developer community, the number of frameworks and platforms included is now 74! Round 6 also adds the often-requested plaintext test with HTTP pipelining (thanks in large part to William Gloze…

One request: I can't help but wonder how JAX-RS / Jersey on Tomcat / Glassfish perform. JAX-RS on Java EE is a serious contender to Spring, and I really want to know how a JAX-RS service (Servlet based) is compared to Spring and to raw Servlets.

Re: Frameworks Round 6

#43
cpoll-cppsp made huge performance gains from round 5 to 6. For example multiple queries went from 1,872 rps in round 5 to 7,252 in round 6. What caused that?

Also what happened to cpoll-cppsp in the plaintext test? Its performance there took a deep dive.

Re: Frameworks Round 6

#44

For such a young language, I'm really surprised by the performance of Go there. Quite awesome.

Round 4 of these tests is what triggered my dive into Go this past month, and it has been a revelation. If you add in the concurrency capabilities, the language simplicity, and nice balance between overly terse syntax (Python one liners...) and overly verbose typed languages (Java), the whole package is even more impressive. The only real thing I think it is missing is a "high productivity" application framework like…

Check out https://github.com/jasondelponte/gokart - Rails style development with go. "The gokart gem by default combines SASS, Coffee Script, Rake, and Sprockets with Go to provide a great development environment. This environment supports test driven development (TDD) with ruby guard and jasmine. There are no external dependencies other than Go and ruby (which I expect you already have...)"

Re: Frameworks Round 6

#45
post #41

An honest confession from someone inspired by these comparisons. I've been following these benchmarks very closely right from round one and ever since I've been waiting to see my favorite framework, Ruby on Rails perform decently to some extent. I waited till the last round to see some improvements and while I DID see some marginal improvements, it wasn't as expected. This then provoked me to do some basic math. Imag…

> For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch????

If I answered yes to literally all of these questions, am I still allowed to use rails?

Re: Frameworks Round 6

#46
post #34

Earlier quoted context omitted.

Round 4 of these tests is what triggered my dive into Go this past month, and it has been a revelation. If you add in the concurrency capabilities, the language simplicity, and nice balance between overly terse syntax (Python one liners...) and overly verbose typed languages (Java), the whole package is even more impressive. The only real thing I think it is missing is a "high productivity" application framework like…

Have you tried modern ML-style languages? e.g: Haskell, OCaml, F#? They also combine the nice conciseness of Python, with great concurrency (better than Go) and a lot more safety (less late night crashes).

With respect, and I say this having experimented with Haskell, OCaml, any many other functional languages as well, just, no. Those languages come at a MUCH higher mental cost than go. Go is wonderful because writing it doesn't feel like I'm attempting a CompSci doctoral thesis. I can also say I have yet to see the Go runtime crash, and I am using in production.

I also challenge your "better concurrency" claim.

Re: Frameworks Round 6

#47

cpoll-cppsp made huge performance gains from round 5 to 6. For example multiple queries went from 1,872 rps in round 5 to 7,252 in round 6. What caused that? Also what happened to cpoll-cppsp in the plaintext test? Its performance there took a deep dive.

I know that the contributor of those tests made several changes. We weren't able to identify why it fails dramatically on the plaintext test, but I suspect it's a combination of the two unique characteristics of that test: the use of HTTP pipelining and higher client-side concurrency. My guess is that the pipelining in particular is causing problems.

Re: Frameworks Round 6

#48
post #32

Earlier quoted context omitted.

Round 4 of these tests is what triggered my dive into Go this past month, and it has been a revelation. If you add in the concurrency capabilities, the language simplicity, and nice balance between overly terse syntax (Python one liners...) and overly verbose typed languages (Java), the whole package is even more impressive. The only real thing I think it is missing is a "high productivity" application framework like…

how about http://robfig.github.io/revel/ ? I just skimmed thru their manual and it seems quite promising

I was just looking at Revel too. But from the homepage:

Development Status: Early adopters only. Pull requests welcome. Development is closing in on the "final" 1.0 design, but the rate of change is still high. Expect to get your hands dirty.

Wouldn't quite appeal for people looking for a Go based Rails yet, but I would have to agree.. it does look very promising.

Re: Frameworks Round 6

#49
How are certain languages so much faster even when pulling from the database? Surely the database would be the bottleneck here? Surely if it's something cool the driver is doing, that could be implemented in other drivers?

Also, why are some frameworks on there a bunch of times (e.g. ASP MVC)?

Re: Frameworks Round 6

#50
post #41

An honest confession from someone inspired by these comparisons. I've been following these benchmarks very closely right from round one and ever since I've been waiting to see my favorite framework, Ruby on Rails perform decently to some extent. I waited till the last round to see some improvements and while I DID see some marginal improvements, it wasn't as expected. This then provoked me to do some basic math. Imag…

> For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch???? If I answered yes to literally all of these questions, am I still allowed to use rails?

Allowed and encouraged! It's good to both have standard tools that not everybody needs to reimplement differently (and often poorly) and to understand how those tools work. Everyone should take the time to get at least highly acquainted with the libraries they depend on. I suppose this was a good way for your parent to learn that lesson, but it's also a good example of why benchmarks aren't everything for frameworks; their design and community matter a lot.
Post reply on HN