It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.
Very true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.
Absolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
The benchmarks for node.js are terribly misleading. The node.js implementations only ever spawn a single process, and thus node is only running on a single core and uses only a single thread. Specifically, the http server example(1), doesn't even bother using the standard library provided Cluster module(2). Cluster is specifically designed for distributing server workloads across multiple cores. All node.js services/…
It says that all the benchmarks are single-threaded and even mentions at the end that you could push performance further with multicore machines.
It doesn't matter anyway, with one thread per core it would be pretty straightforward to scale in beefier machines.
The benchmarks for node.js are terribly misleading. The node.js implementations only ever spawn a single process, and thus node is only running on a single core and uses only a single thread. Specifically, the http server example(1), doesn't even bother using the standard library provided Cluster module(2). Cluster is specifically designed for distributing server workloads across multiple cores. All node.js services/…
Since this is a benchmark of eventloop based frameworks, it makes sense to only spawn a single eventloop and test against that. I looked through the code for the python servers and they are all configured for a single event loop, making this a comparison on equal footing.
Yes it's true you normally run multiple node processes in production, but you likewise normally run multiple asyncio/tornado/twisted processes in production as well. I don't see it as a big deal, or misleading to compare them in this sense.
1) I don't know :( I've answered a similar question in this thread with a couple of guesses. 2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2. 3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there a…
> 2) No, they have a different architecture Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect). If you are keen, this has the potential to be the kill…
There is already http://www.tornadoweb.org/en/stable/ which is python3 compatible and shipped in production software across lots of companies (last I heard hipmunk and quora uses it)
I'm actually thinking about writing such a framework :) I'll call it "spin".
You totally should. I think people are hungry for a next gen async web framework... And if it leverages Python 3, then so much the better. Flask cannot go here even if it wants because its core philosophy is to be WSGI compliant. You don't necessarily have to adhere to that. Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database…
> Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database interaction until it's too late.
I strongly advise against this. One of the reasons Flask is so attractive is the fact that it does not enforce any database on you.
Thanks to its decoupled design you can use it purely as a routing library, which is great! Letting the framework decide something important as the database is a bad idea. [1]
> However, the performance bottleneck in aiohttp turned out to be its HTTP parser, which is so slow, that it matters very little how fast the underlying I/O library is. This is exactly the same observation that motivated the Mongrel web server for Ruby, 10 years ago this year. "Mongrel is a small library that provides a very fast HTTP 1.1 server for Ruby web applications. [...] What makes Mongrel so fast is the caref…
It's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.
These days you would probably want to write the parser part in Rust, with a small amount of unsafe code to implement a C-compatible API that could then be called from Python, or wherever. I did this for some regexp-based log parsing code written in Python, and saw a considerable (2-3x) performance win. The main outstanding issue is that Rust isn't as easy to distribute as C to random end users (e.g. it likely requires the user to have rustc installed for |pip install| to work, which is unlikely and not always possible through standard package managers).
Also note that David Beazley (google him if you're not aware) has a competitor to asyncio, which arguably is also a direct competitor to this called curio: http://curio.readthedocs.io/en/latest/ The caveat is that it uses the new async/await coroutine bits that just landed in Python 3.5, so it only works with Python 3.5+. He also gave a talk on concurrency in python recently at last year's PyCon: https://www.youtube.…
Unless I'm mistaken, the decorator @asyncio.coroutine() is equivalent to async def and yield from is functionally a drop-in for await, so you should be able to use it with at least 3.4, maybe 3.3. Not that that's much better though.
That makes it functionally equivalent to uvloop (which from my understanding is a drop-in replacement for the built-in asyncio eventloop).
The benchmarks for node.js are terribly misleading. The node.js implementations only ever spawn a single process, and thus node is only running on a single core and uses only a single thread. Specifically, the http server example(1), doesn't even bother using the standard library provided Cluster module(2). Cluster is specifically designed for distributing server workloads across multiple cores. All node.js services/…
Do the other benchmarkside spawn multiple processes?
No, even Go is explicitly configured to only use one scheduler:
> We use Python 3.5, and all servers are single-threaded. Additionally, we use GOMAXPROCS=1 for Go code, nodejs does not use cluster, and all Python servers are single-process.