61–70 of 77 posts
This is great for python, a proven web framework that was stressed on friend feed. Thanks ff team! I have been torn on my next python server project between web2py, cherrypy and django and I think I just decided.
For the record. They tested web.py not web2py. It is not the same thing and they are completely unrelated.
Awesome to see more Python webservers coming to light, and fast ones at that. One observation though; anyone else notice that the bar chart is a bit misleading? It compares running Tornado in 4 processes behind nginx to running Django behind Apache and Cherrypy as a single Python process. As a fellow coworker put it, "With 4 extra processes running of course it will be 4 times as fast." I'm not opposed to this kind o…
Apache/mod_wsgi creates multiple processes (or can). The issue is that Facebook didn't tell us what configuration settings they used. Did they limit the number of processes? Did they not have it start up enough at the start? It's clear that Facebook wasn't using a single Apache/mod_wsgi process because that wouldn't get close to 2,000 requests per second. I'm sure they gave it the number of processes that were approp…
IApache's only saving grace is that you can handle everything monolithically (albeit with mediocrity).
Earlier quoted context omitted.
Thanks. So to clarify, this is just the same as what lighttpd phrases as a 'select()-/poll()-/epoll() based web server'? It seems that the main advantage of this is that you have one thread manging many sockets. I am a bit surprised that blocking kthreads would be so much slower relatively. What causes the slowness? Context switching? Additional stack memory usage?
You guessed right. Context switching is expensive, and the default stack size default is in the megabyte range. So if you want to have 10K connections open, it takes about 10GB memory. You can of course shrink the stack size, but you have to measure your programs stack usage before doing that and it is quite cumbersome. With an event driven architecture you only have to hold the session information per connection in…
Earlier quoted context omitted.
Thanks. So to clarify, this is just the same as what lighttpd phrases as a 'select()-/poll()-/epoll() based web server'? It seems that the main advantage of this is that you have one thread manging many sockets. I am a bit surprised that blocking kthreads would be so much slower relatively. What causes the slowness? Context switching? Additional stack memory usage?
I don't think threads are slower or less scalable anymore. The OS threading systems have improved a lot over the last four years.
What exactly does 'non-blocking' mean in the context of a webserver?
Earlier quoted context omitted.
If you ran a single CherryPy instance, I don't see how that is possible. The Python global interpreter lock prevents a single process from making use of multiple cores.
No, it doesn't. I/O bound tasks can heavily use multiple cores just fine using threads.
Even cpu bound tasks can release the GIL to do processing(for non python api tasks).
Earlier quoted context omitted.
You guessed right. Context switching is expensive, and the default stack size default is in the megabyte range. So if you want to have 10K connections open, it takes about 10GB memory. You can of course shrink the stack size, but you have to measure your programs stack usage before doing that and it is quite cumbersome. With an event driven architecture you only have to hold the session information per connection in…
I believe this is one of the advantages of 64 bit architectures. That meg of stack size is just a reservation, it's not actually used, unless your thread really needs it. On a 64 bit machine there is plenty of space to allocate.
Anyways, I was talking about C, where the stack is allocated when the thread is created and you cannot resize it later, nor it will grow automatically.
With dynamic languages it is different, but don't expect handling that much connections with them either.
Earlier quoted context omitted.
Thanks. So to clarify, this is just the same as what lighttpd phrases as a 'select()-/poll()-/epoll() based web server'? It seems that the main advantage of this is that you have one thread manging many sockets. I am a bit surprised that blocking kthreads would be so much slower relatively. What causes the slowness? Context switching? Additional stack memory usage?
You guessed right. Context switching is expensive, and the default stack size default is in the megabyte range. So if you want to have 10K connections open, it takes about 10GB memory. You can of course shrink the stack size, but you have to measure your programs stack usage before doing that and it is quite cumbersome. With an event driven architecture you only have to hold the session information per connection in…
Typically an output buffer should be able to hold the complete production for a client, if you limit it to say 4K you will be unable to process a request in one go.
I've been looking for a good framework to begin serious web development on for someone coming from a decade-long career in desktop development (see the discussion at http://news.ycombinator.com/item?id=808191 ), and decided this release co-incides nicely with my newly-started quest and gave it a shot. I realize Tornado isn't any different from the other Python frameworks with regards to coding style, etc. but the the…
What exactly does 'non-blocking' mean in the context of a webserver?
Non-blocking sockets. Calls to socket.read() can return no data. It's non-blocking because the program is not stuck waiting for data to be returned.