Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

Post reply on HN
Print view View original

191–200 of 358 posts

Re: Web Framework Benchmarks Round 4

#191

Post by ricardobeat »

oberhamsi wrote: post #161

I implemented the Ringo app for this benchmark and of course ran it against Node and a couple of others to see how we would perform in this neighborhood before I opened the pull request. And since that day I've been wondering: why does NodeJs (=V8 JS engine in C) talking to MongoDB have higher response times and latency than Ringo (=Rhino JS engine on JVM) talking to MySQL. The only thing where Node beats us JVM guys…

Node favours concurrency over raw speed; calls deferred with process.nextTick and callbacks end up costing time, in exchange for better concurrency. I think a blocking driver could leave Ringo in the dust, but it would be useless.

Re: Web Framework Benchmarks Round 4

#192

Post by raylu »

peeters wrote: post #75
hippich wrote: post #57

Whenever I hear "Java" I also get association "slow". But looking into this list - java web frameworks doing incredible job!

Java gets that wrap from originally being slow to execute, and also having a huge up-front cost to spin up a VM. The first isn't true any more: the Java VM competes with native code on most benchmarks, and due to its ability to perform runtime optimizations, can occasionally outperform native code. The second doesn't matter at all for web servers. The cost of starting up the web server is tertiary to uptime and perfo…

I agree with everything you've said here, but I'd like to add something about startup time.

If it takes you 5s to start up your server, that's a lot of time you've added to each development iteration. Make a change, restart the server, wait 5s, see if it works/check debug output.

Re: Web Framework Benchmarks Round 4

#193

Post by zenlikethat »

landr0id wrote: post #163
gtaylor wrote: post #63

Earlier quoted context omitted.

As someone who is also really enjoying Go, I think you need to add a huge, gigantic disclaimer before making a statement like this: Go's ecosystem of web development packages is in its infancy. You're not going to find any super-well-documented, super mature/stable web frameworks (thought a few are showing great promise). and some of the individual components (for example, Gorilla) are looking very good, but still ha…

While we're on the subject... Has anyone tried out Revel[.]? If so, what are your thoughts on it? [.] https://github.com/robfig/revel

(Disclaimer: I'm pretty new to Go). I tried Revel out a little while ago, and found it awesome at first. It felt like it could have the potential to dramatically speed up the development of Go web apps. I like the server and hotswapping features a lot, for instance.

But at the moment unfortunately I don't think it's very mature. Support for interacting with the DB, arguably the most important part of a web application, is pretty lacking IMO for something that otherwise wants to be an end-to-end solution.

In one of the examples I noticed that all kinds of interaction with the DB was being done in the Controller, not the Model. Which just seems wrong to me (at first I thought it was a "Play" framework thing, since Revel is modeled on that, but Play uses Hibernate for an ORM in its models). Also, you'll have to roll your own support for interacting with the DB using, say, gorp: https://github.com/coopernurse/gorp

That being said, robfig seems like a really cool dude, and he was responsive on github when I needed some help. The documentation is pretty great too.

Re: Web Framework Benchmarks Round 4

#194

Post by desireco42 »

I have one more question, I see a number of really fat frameworks on top, if rails is faster, then it must be because it is heavily optimized and people who made it are super smart, but for example codeigniter is above slim, kohana is above fuel, like twice the speed for Fortunes test. This isn't what I would expect at all, I would also expect Rails to be below php frameworks for speed alone.

Did you use orm when it was available? Or just used raw query?

This is in line with my previous query about more usage scenarios (and thank you for fortunes test) who would be like we use them usually.

My point is, that the same reason why we use framework instead of raw language, we use orm instead of direct sql.

Re: Web Framework Benchmarks Round 4

#195

Post by binarysoul »

candybar wrote: post #188
dragonwriter wrote: post #182

Earlier quoted context omitted.

> Many people choose Ruby and figure, given that premature optimization is the root of all evil, they'll optimize later if needed. True. > That's like choosing between a farm tractor or a ferrari - and figuring if the tractor doesn't perform up to snuff, we'll add a spoiler Its really not like that at all, because programming languages aren't like vehicles. Particularly, with Ruby, on typical method of optimization i…

I keep hearing about this, but do people really rewrite performance-critical parts of their web apps in C? Even if it happens to be part of some third-party library? And maintain a fork? What if that performance-critical part is dependent on other parts in a non-trivial way? It seems that an unanticipated replacement of some core functionality with a C library may involve a major rewrite and most Ruby teams may not h…

In many cases it is as easy as relying on a gem instead of doing it yourself. Where the gem contains a c extension. Say resizing images with chunkypng vs mini_magick

Re: Web Framework Benchmarks Round 4

#196

Post by oberhamsi »

freework wrote: post #119
trailfox wrote: post #91

Earlier quoted context omitted.

For many systems performance matters. Claiming that there is no difference between Ruby and C is just sticking your head in the sand.

You can build an application in Ruby, deploy it onto 15 machines, and it will outperform the same application written in C and deployed to only one machine. Performance is more of a function of the underlying hardware than the language used to build it.

If you split a single app instance onto 15 machines, you will lose some efficiency due to network communication unless those 15 instances can work isolated without any shared data (sessions). That may not be much but worst case: you have to write that inter-machine sync code.

Re: Web Framework Benchmarks Round 4

#197

Post by jbverschoor »

raylu wrote: post #192
peeters wrote: post #75

Earlier quoted context omitted.

Java gets that wrap from originally being slow to execute, and also having a huge up-front cost to spin up a VM. The first isn't true any more: the Java VM competes with native code on most benchmarks, and due to its ability to perform runtime optimizations, can occasionally outperform native code. The second doesn't matter at all for web servers. The cost of starting up the web server is tertiary to uptime and perfo…

I agree with everything you've said here, but I'd like to add something about startup time. If it takes you 5s to start up your server, that's a lot of time you've added to each development iteration. Make a change, restart the server, wait 5s, see if it works/check debug output.

Ruby or Python are even worse (slow / big) than java these days.

I'm a ruby guy btw..

Re: Web Framework Benchmarks Round 4

#198

Post by oberhamsi »

desireco42 wrote: post #194

I have one more question, I see a number of really fat frameworks on top, if rails is faster, then it must be because it is heavily optimized and people who made it are super smart, but for example codeigniter is above slim, kohana is above fuel, like twice the speed for Fortunes test. This isn't what I would expect at all, I would also expect Rails to be below php frameworks for speed alone. Did you use orm when it…

the source is available. as a first guess: if an ORM is the typical way to do it in an env then it was used.

Re: Web Framework Benchmarks Round 4

#199

Post by sbov »

candybar wrote: post #188
dragonwriter wrote: post #182

Earlier quoted context omitted.

> Many people choose Ruby and figure, given that premature optimization is the root of all evil, they'll optimize later if needed. True. > That's like choosing between a farm tractor or a ferrari - and figuring if the tractor doesn't perform up to snuff, we'll add a spoiler Its really not like that at all, because programming languages aren't like vehicles. Particularly, with Ruby, on typical method of optimization i…

I keep hearing about this, but do people really rewrite performance-critical parts of their web apps in C? Even if it happens to be part of some third-party library? And maintain a fork? What if that performance-critical part is dependent on other parts in a non-trivial way? It seems that an unanticipated replacement of some core functionality with a C library may involve a major rewrite and most Ruby teams may not h…

I've never heard of it for webapps. They usually just buy more instances/servers.

It would be interesting to see the profile of some these benchmarks for the various frameworks to see where the bottleneck is.

Re: Web Framework Benchmarks Round 4

#200

Post by jakejake »

bhauer wrote: post #113
saosebastiao wrote: post #15

I love what you guys are doing. This is by far the most comprehensive (in terms of number of frameworks) comparison of web frameworks. I also am a big fan of the new filtering metadata. However, I'm starting to think that all of the advocates of various frameworks are now conspiring independently to make this comparison meaningless...any framework (except Cake for some reason) can be superoptimized towards a small se…

Hi saosebastiao, As Pat points out, we definitely look forward to implementing some more computationally-intense request types in the future. This round does include the first server-side template test. We'd like to hear the community's opinions about more tests. That said, I feel most of the frameworks' implementations of the existing tests are not cheating. Our objective in this project is to measure every framewor…

I would like to pile my thanks onto this list as well. I'm the author of Phreeze and I can say that I'm grateful that fairness is being encouraged. There is certainly glory in ranking well on any benchmark and I have to admit, as I was implementing the tests in Phreeze, I saw many opportunities to "cheat." For example, skipping the framework routing, not using the "proper" way to communicate between the layers, etc and substituting things with "raw" code would have potential to skew the result. I feel that would be missing the entire point of a benchmark, so I'm glad that is being considered.

I can also say that this benchmark inspired me to take a hard look at class loading and I was able to make some improvements to the framework's efficiency in general. So, in a way, I did some tuning - not for the benchmark, but rather as a result of the benchmark. Thanks to this benchmark all Phreeze users will gain a little performance.

I would also like to suggest a test idea. I think the biggest challenge for frameworks comes into play when you have to do table joins. Something like looping through all purchase orders and displaying the customer name from a 2nd table - that would be a very real-world type of test. I think foreign key type of queries are more telling about an ORM than a single table query.

Thanks again!

Post reply on HN