Understanding the code inside Tornado, the asynchronous web server
1–10 of 43 posts
Re: Understanding the code inside Tornado, the asynchronous web server
#2> I also feel obliged to talk a bit on nonblocking IO or asynchronous IO (AIO)
Wait, that is not correct. His IO is blocking. He gets a notification when data is ready, from the select/poll/epoll (the asynchronous part), but when IO is actually performed (read, write, recv, etc), the operation it is still happening in the main thread in user space, and it blocks it.
Currently only the file system (I think) has truly asynchronous and non-blocking IO. It is provided by the aio_* set of system calls and is has been sort of an exotic beast (it is not that popular).
Here is a good chart of the possible IO types and their combinations:
http://www.ibm.com/developerworks/linux/library/l-async/
And of course:
Re: Understanding the code inside Tornado, the asynchronous web server
#3> I also feel obliged to talk a bit on nonblocking IO or asynchronous IO (AIO) Wait, that is not correct. His IO is blocking. He gets a notification when data is ready, from the select/poll/epoll (the asynchronous part), but when IO is actually performed (read, write, recv, etc), the operation it is still happening in the main thread in user space, and it blocks it. Currently only the file system (I think) has truly…
No the network sockets are set to non block and will return E_AGAIN once you have emptied the kernel buffer. Thats still non blocking.
I dont think he is using aio_* which Linux does not really implement usefully.
Re: Understanding the code inside Tornado, the asynchronous web server
#4> I also feel obliged to talk a bit on nonblocking IO or asynchronous IO (AIO) Wait, that is not correct. His IO is blocking. He gets a notification when data is ready, from the select/poll/epoll (the asynchronous part), but when IO is actually performed (read, write, recv, etc), the operation it is still happening in the main thread in user space, and it blocks it. Currently only the file system (I think) has truly…
No the network sockets are set to non block and will return E_AGAIN once you have emptied the kernel buffer. Thats still non blocking. I dont think he is using aio_* which Linux does not really implement usefully.
Python doesn't provide an API to aio_* because they're basically useless (on Linux at least).
Re: Understanding the code inside Tornado, the asynchronous web server
#5> I also feel obliged to talk a bit on nonblocking IO or asynchronous IO (AIO) Wait, that is not correct. His IO is blocking. He gets a notification when data is ready, from the select/poll/epoll (the asynchronous part), but when IO is actually performed (read, write, recv, etc), the operation it is still happening in the main thread in user space, and it blocks it. Currently only the file system (I think) has truly…
No the network sockets are set to non block and will return E_AGAIN once you have emptied the kernel buffer. Thats still non blocking. I dont think he is using aio_* which Linux does not really implement usefully.
No. They would return E_AGAIN after the data has been read or written in blocking mode. If select comes back and says "read is ready on socket 4" then user code does read on socket 4 and gets (say) 4K of data followed by E_AGAIN. Getting that 4K of data happens in a blocking mode. To make it non-blocking you would provide a pointer to your user space buffer to the kernel and it would determine when IO is ready _and_ proceed to put that data in the buffer.
Re: Understanding the code inside Tornado, the asynchronous web server
#6Earlier quoted context omitted.
No the network sockets are set to non block and will return E_AGAIN once you have emptied the kernel buffer. Thats still non blocking. I dont think he is using aio_* which Linux does not really implement usefully.
No. They would return E_AGAIN after the data has been read or written in blocking mode. If select comes back and says "read is ready on socket 4" then user code does read on socket 4 and gets (say) 4K of data followed by E_AGAIN. Getting that 4K of data happens in a blocking mode. To make it non-blocking you would provide a pointer to your user space buffer to the kernel and it would determine when IO is ready _and_…
The way you are using the term "blocking" is non-standard and not that useful. You are calling the syscall "blocking" just because it performs its work inline (in this case, copying a kernel buffer into a user-space buffer). By this definition, every syscall is blocking, even aio_read() because even aio_read() performs some work inline (namely enqueing a read request).
In common usage, an I/O operation is considered "blocking" if the system call does not return until data is available. If on the other hand you set O_NONBLOCK on a fd, you will get EAGAIN when no data is available, so clearly that is the accepted meaning of "nonblocking."
Re: Understanding the code inside Tornado, the asynchronous web server
#7Personally, I would be more interested in understanding the code inside Tornado, a column of air. :-)
Re: Understanding the code inside Tornado, the asynchronous web server
#8> Lets say you have 20 threads. You improved performance 20 times, so the rate is now 4 request per second. Still, way too small. You can keep throwing threads at the problem, but threads are expensive in terms of memory usage and scheduling. I doubt you’ll ever reach hundreds of requests per second this way.
Let's say I'm using Java instead of Python. Lets say I use a lot more threads than 20. I will reach thousands of requests per second this way. There are real problems with that approach, and event-driven approaches have genuine advantages, but can people please stop straight-up lying and saying that you can't just throw a few thousand threads at a problem, because you can, and people do.
Re: Understanding the code inside Tornado, the asynchronous web server
#9> Lets say you have 20 threads. You improved performance 20 times, so the rate is now 4 request per second. Still, way too small. You can keep throwing threads at the problem, but threads are expensive in terms of memory usage and scheduling. I doubt you’ll ever reach hundreds of requests per second this way. Let's say I'm using Java instead of Python. Lets say I use a lot more threads than 20. I will reach thousands…
I agree with you mostly, and I think there is a misunderstanding of the terms. It's possible to serve large amounts of requests with threads as well as with event loops, but there is a distinction between the two on how they consume resources. The author is wrong in assuming threads will not scale, as they will, but a Java web framework using threads will eat up hundreds of megs under heavy load, while an event loop will keep on churning away with progressively longer response times. It's not so much a case of lying as much not knowing when to stop presuming instead of investigating the actual facts.
Re: Understanding the code inside Tornado, the asynchronous web server
#10> Lets say you have 20 threads. You improved performance 20 times, so the rate is now 4 request per second. Still, way too small. You can keep throwing threads at the problem, but threads are expensive in terms of memory usage and scheduling. I doubt you’ll ever reach hundreds of requests per second this way. Let's say I'm using Java instead of Python. Lets say I use a lot more threads than 20. I will reach thousands…
Yeah, whenever people say threads don't work, point them to Varnish. It uses threads intelligently and everyone agrees it's fast and awesome.