Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks

techempower.com

Post reply on HN
Print view View original

311–320 of 415 posts

Re: Web Framework Benchmarks

#311

Post by smandou »

bilbo0s wrote: post #128
jondot wrote: post #26

Earlier quoted context omitted.

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

I'm not sure many people would use webgo in real life. I don't know... maybe some people... certainly not pros. Also, the 1.0.3 thing is probably dragging on the numbers a bit. 1.1 would boost it a little. Not enough to get it into the top tier... but a little. Also, for Vert.x, they seem to be only running one verticle. Which would never happen in real life. Play could be optimized a bit... but not much. What they h…

The Play app uses MySql in a blocking way, while Nodejs uses mongo. It's not comparable.

Re: Web Framework Benchmarks

#312

Post by spartango »

vanderZwan wrote: post #227
k__ wrote: post #171

Earlier quoted context omitted.

What's wrong with that guy?

It's a pretty serious problem with how we benchmark though. EDIT: when looking up what I vaguely remembered I somehow managed to come across a similar article that was published just today[1], even though I was referring to an older one[2] which was about microstuttering (basically: a high standard deviation in frame rate). The point still stands - in fact it applies to both cases in somewhat different ways. To give…

Indeed, it's important to look not just at the average performance and performance extremes, but also the distribution of performance.

Standard deviation helps with this. Also, often times looking at the latency at the 50, 90, and 99th percentiles is valuable as you can see events that would make your users unhappy. They're a very tangible series of metrics.

Re: Web Framework Benchmarks

#313

Post by pifflesnort »

bradrydzewski wrote: post #258
jondot wrote: post #26

Earlier quoted context omitted.

I think this is a much needed and excellent point to make. Just take a look at how Go dips down when using Webgo.

I used Go's benchmarking tool to compare raw routing performance of various frameworks. The handlers all return a simple "Hello World" string. Here are the results: PASS Benchmark_Routes 100000 13945 ns/op Benchmark_Pat 500000 6068 ns/op Benchmark_GorillaHandler 200000 11042 ns/op Benchmark_Webgo 100000 26350 ns/op ok github.com/bradrydzewski/routes/bench 12.605s I then ran the same benchmark, but this time I modifie…

rorr, you appear to be hellbanned. Here's your comment, since it seemed like a reasonable one:

> Now, imagine I created a third benchmark that did something more complex, like executing a database query and serving the results using the html/template package. There would be a negligible difference in performance across frameworks because routing is not going to be your bottleneck. If you're performing a DB query on every request, you're doing something wrong. In the real world your app will check Memcached, and if there's a cached response, it will return it. Thus making the framework performance quite important.

Re: Web Framework Benchmarks

#316

Post by rombdn »

saym wrote: post #100
rombdn wrote: post #87

Very interesting! But isn't there a disavantage for node.js considering it's single threaded? Did you cluster it to use every core on the box ? ( http://stackoverflow.com/a/8685968/1909827 - "Scaling throughput on a webservice")

From the OP's response to a similar question: >We attempted to take advantage of threading/multiple-processes as best we could (see the nodejs code for an example of using the cluster module). But we suspect there are additional areas of improvement here.

Thanks

Re: Web Framework Benchmarks

#317

Post by bhauer »

asadkn wrote: post #286
bhauer wrote: post #92

Earlier quoted context omitted.

It's funny you should put this together because in an earlier draft of this blog entry I had created a tongue-in-cheek unit to express the average cost per additional line of code. Based on our Cake PHP numbers, I wanted to describe PHP as having the highest average cost per line of code. But we dropped this because I felt it ultimately wasn't fair to say that based on the limited data we had and it could be easily i…

Good thing you opted against sensational journalism. About the cost per additional line of code for PHP, it mainly comes from not having an opcode cache and having to load and interpret files on every visit. mod_php was and will always be trash. I commented earlier about it too. In case of Ruby, and talking about Rails, even when using Passenger, the rails app is cached via a spawn server. That's not the case with PH…

We've received a lot of great feedback already and even several pull requests. Speaking of, we want to express a emphatic "Thank you" to everyone who has submitted pull requests!

We're hoping to post a follow up in about a week with revised numbers based on the pull requests and any other suggested tweaks we have time to factor in. We're eager to see the outcome of the various changes.

Re: Web Framework Benchmarks

#318

Post by bhauer »

knappador wrote: post #298
rozap wrote: post #207

Earlier quoted context omitted.

There are a lot of variables and tweaking that can be done, and it would be nearly impossible to optimize each. Similarly, I was wondering what sort of an effect connection pooling would have, as the out of the box django distribution doesn't do that. It really didn't perform too well in their tests.

At LEAST gevent with session write-through caching, psycopg2pool, Postgres SQL (hello, excellent South support?), no unnecessary middlewares or applications that rely on them (if it's a speed oriented use of Django, we're hosting on a specific API sub-domain, right?). At most, THEN you tune the settings to have an optimum number of Postgres threads staying alive and tweak some gunicorn/nginx max connections parameter…

Thanks for your pull requests, knappador. We will try to get a revised post out in roughly a week or so with as many of the tweaks we've received (as pulls and tips) as we can muster.

Re: Web Framework Benchmarks

#319

Post by bhauer »

jaddison wrote: post #173
cridenour wrote: post #134

I'm curious to see Django results when using the gevent worker for gunicorn. For these type of quick JSON calls, you can see huge performance increases.

Along with gevent, it would be good to throw in psycogreen to improve DB (well, postgresql) evented connections.

It looks as if we've got a Github pull request including these changes, so we'll be able to revise the Python-Django numbers soon.

Re: Web Framework Benchmarks

#320

Post by bhauer »

larkinrichards wrote: post #292

I roughed up a flask benchmark but I haven't been able to test locally. If you want to help, check out this pull request: https://github.com/TechEmpower/FrameworkBenchmarks/pull/14/f...

Awesome, thank you! We'll aim to get this incorporated into a follow up post soon.
Post reply on HN