11–20 of 38 posts
Summary: * Mean response time -- Twisted.Web better than Tornado Average performances for Concurrent request: * @100 -- TW:Tor::0.50s:0.70s . TW better than Tor * @500 -- TW:TOr:: 3s:3.5s . TW slightly better than Tor * @1000 -- TW:Tor:: 5s:6.5s . TW better than Tor
Earlier quoted context omitted.
I did split the machines. The driver (the one running the test) machine is completely different than the webserver running twisted.web or tornado.
But do you have any idea what else was running on those virtual machines and what kind of overhead that gave ?
Earlier quoted context omitted.
Given how common it is for production applications to be running in the cloud, this test is both valid and useful. Therefore these tests are much more meaningful than the first.
I disagree. If you run a test on a virtual machine then you are testing a lot of other software besides the software under test so the picture gets distorted. For instance, in the graphs that are in the report you can see a bunch of drops to '0' traffic, these suggest serious problems with the software under test. But knowing that the tests were done in a VM environment it is possible that the cause of these drops is…
I mean, I think I get your point, in that this benchmark doesn't apply to those running bare metal... but isn't the obvious counterpoint that this is an especially useful benchmark for those running in a VM?
Earlier quoted context omitted.
I disagree. If you run a test on a virtual machine then you are testing a lot of other software besides the software under test so the picture gets distorted. For instance, in the graphs that are in the report you can see a bunch of drops to '0' traffic, these suggest serious problems with the software under test. But knowing that the tests were done in a VM environment it is possible that the cause of these drops is…
But if you're testing them both in a VM, wouldn't you consider this a valid performance benchmark for those looking to run their instance in a cloud VM? I mean, I think I get your point, in that this benchmark doesn't apply to those running bare metal... but isn't the obvious counterpoint that this is an especially useful benchmark for those running in a VM?
Summary: * Mean response time -- Twisted.Web better than Tornado Average performances for Concurrent request: * @100 -- TW:Tor::0.50s:0.70s . TW better than Tor * @500 -- TW:TOr:: 3s:3.5s . TW slightly better than Tor * @1000 -- TW:Tor:: 5s:6.5s . TW better than Tor
* Where are you getting your estimates from? The numbers on the spreadsheet in the article do not match your rounded summaries.
* Twisted is faster at the same proportions for all the concurrent request numbers. Your summary makes it sounds like it's only slightly better at 500, but significantly better at all other request numbers. In fact, it's about 120% faster across the board.
Since yours is the top rated comment, I think you should ensure that it's accurate.
Earlier quoted context omitted.
But do you have any idea what else was running on those virtual machines and what kind of overhead that gave ?
Yes, I know exactly what was running on those virtual machines. They were purpose built for this test, nothing else was creating load on those servers.
Or are you sure that the machines these virtual machines were running on were otherwise completely idle ?
And if they were idle then why bother with the virtual machines, then you could have just run your tests directly bypassing the whole VM overhead.
I question the relevance of these tests. With all the attention being given to performance, everyone seems to be overlooking the fact that Tornado provides zero support for truly asynchronous web applications written for it. If you have any kind of long-running task to perform during a web request, you have to roll your own way of dealing with that task. To make it worse, the demos included with tornado that make use…
Glyph made a really good point, tornado comes with zero automated tests... Ultimately after conducting these tests and listening to the discussion that has taken place, I am personally glad that we're writing our code with twisted.web
I don't think these tests are much more meaningful than the first. They still use virtual machines, which give you all kinds of invisible overhead. Even if that were not the case you should definitely split the machine that does the tests off from the machine that is being tested otherwise you get contention between the test program and the programs being tested (and java being quite cpu intensive running both tests…
I don't think these tests are much more meaningful than the first. They still use virtual machines, which give you all kinds of invisible overhead. Even if that were not the case you should definitely split the machine that does the tests off from the machine that is being tested otherwise you get contention between the test program and the programs being tested (and java being quite cpu intensive running both tests…
Also, they're testing Hello World. IMHO that makes the tests close to meaningless.
#1. because it's what Bret from Friendfeed used in his testing #2. it tests the core framework code, anything more is testing code I write.
I question the relevance of these tests. With all the attention being given to performance, everyone seems to be overlooking the fact that Tornado provides zero support for truly asynchronous web applications written for it. If you have any kind of long-running task to perform during a web request, you have to roll your own way of dealing with that task. To make it worse, the demos included with tornado that make use…