31–40 of 132 posts
I'm the dev behind uvloop. AMA.
It looks like uvloop requires Python 3.5. How practical is it to create a fork for companies stuck on Python 2.7?
I'm the dev behind uvloop. AMA.
1) What makes uvloop which is based on libuv 2x faster than node.js which is also based on libuv? 2) Can uvloop be used with frameworks like flask or django? 3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?
Earlier quoted context omitted.
Building on to this, how does it compare to raw libuv in c?Personally, I'm not surprised that python (especially cython) is faster than node in this case, but I still need to see how much less overhead there is to node.
> Building on to this, how does it compare to raw libuv in c? Building something in C is very hard. uvloop wraps all libuv primitives in Python objects which know how to manage the memory safely (i.e. not to "free" something before libuv is done with it). So development time wise, uvloop is much better. As for the performance, I guess you'd be able to squeeze another 5-15% if you write the echo server in C. > I'm not…
Yeah, I definitely get that. I'm just trying to see the smaller picture here.
>Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than Python.
Ah, my bad for not knowing the difference between Cython and CPython. It seems to me, then, that this isn't really a fair comparison to node, is it? Naturally a statically typed language is going to be faster than a dynamic one. Good on you for including a comparison with Go, though.
What's the point? No Python 2.7
I'm the dev behind uvloop. AMA.
I'm working on a Python CLI that uses asyncio/aiohttp to make and process requests to a 3rd party API. Anyways, I ran into the 10,000 socket problem today and ended up using a semaphore, that actually boosted the overall performance. Why is that? Is it just because the CPU is overwhelmed otherwise?
Or maybe it's a simple problem of aiohttp performance -- as shown in the blog post, its HTTP parser is a bit slow.
In general, I'd recommend to use a fewer number of sockets and implement some pipelining of API requests.
What's the point? No Python 2.7
Unless you're working with legacy project, in which case you wouldn't be replacing the lib for such stuff anyway, I don't see any reason why you would want to use 2.7.x over 3.x unless you desperately need something like gevent.
It is interesting that uvloop-streams is almost identical to gevent in performance. Gevent is based on libev, the project libuv was forked from. What exactly is the -streams addition that makes uvloop-streams perform so much worse than plain uvloop?
Wow. What's the intuition for why python on top of libuv is 2x faster than node on top of libuv?
Two things I'd check first: 1. The benchmarks make servers generate a huge number of objects, so maybe, the GC is under too much pressure. 2. Another possibility is that the v8 JIT can't optimize some JS code in nodejs, or does a poor job. That said, only careful profiling can answer your question :)
socket.pipe(socket);Earlier quoted context omitted.
1) What makes uvloop which is based on libuv 2x faster than node.js which is also based on libuv? 2) Can uvloop be used with frameworks like flask or django? 3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?
1) I don't know :( I've answered a similar question in this thread with a couple of guesses. 2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2. 3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there a…
Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect).
If you are keen, this has the potential to be the killer application for Python 3.
Also note that David Beazley (google him if you're not aware) has a competitor to asyncio, which arguably is also a direct competitor to this called curio: http://curio.readthedocs.io/en/latest/ The caveat is that it uses the new async/await coroutine bits that just landed in Python 3.5, so it only works with Python 3.5+. He also gave a talk on concurrency in python recently at last year's PyCon: https://www.youtube.…
There are benchmarks of curio in the blog post, FWIW.