Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async I/O for Python 3

dropbox.com

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Async I/O for Python 3

#41

Post by codygman »

rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

Unicode support, concurrency/parallelism, and monkey patching were very big reasons for me using Go rather than python.

Re: Async I/O for Python 3

#42

Post by pjscott »

hosay123 wrote: post #12
rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

Sane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran,…

If your bizarre ancient internal thing doesn't need to do async I/O, then don't import it monkey-patched. I use Eventlet heavily in production, and this tends to be pretty easy.

Re: Async I/O for Python 3

#43

Post by rdw »

hosay123 wrote: post #12
rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

Sane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran,…

As someone who's contributed to Eventlet, I've always felt that the only way that it could get over this scary hurdle (and it is legitimately scary) is for it to be integrated within Python itself. Almost all of the weird problems come from fighting with the baked-in assumptions of the Python runtime. Eventlet does try to alleviate the scariness a little bit by allowing you to import "greened" modules rather than changing the global versions, but that has its own problems.

If it were integrated with Python, there would be no monkeypatching, no special magic, it would be just how things work. That said, I'm not at all surprised that Guido doesn't favor a coroutine-based solution; his opposition to general coroutines is as famous as his opposition to anonymous functions. (to clarify: I don't think the @coroutine decorator creates "real" coroutines, any more than generators were already coroutines)

Re: Async I/O for Python 3

#44

Post by nashequilibrium »

rdtsc wrote: post #32
craigyk wrote: post #27

Earlier quoted context omitted.

Yeah, if you're going to have to do async to be performant than it better be pretty pervasive throughout all the libraries. Bonus points if the language supports syntax to make async easier as well. Node is beating out Python for server stuff not simply because it is "async", but because it is so much FASTER. The speed of V8 vs. CPython is a big part of that. In fact, vanilla JS doesn't have much to make async progra…

I think there is another issue here. Python world has watched as Node.js has been eating its lunch on the server side and they decided, ah, surely that is because Node.js has async, if we add that too, everyone will love Python again and come crawling back. They are not saying, I think that is written between the lines. Except one thing, as you pointed out, people use Node.js -- 1) it is JS 2) V8 is fast.

The reason why you getting so many downvotes is because your comment is not just silly but flat wrong. Twisted, Tornado were around before nodejs. There are also async frameworks that make writing async code the same as synchronous code. I like tornado but I am using nodejs for my current app because of the libraries. This is where nodejs really shines, the community and libraries are awesome. Twisted has a lot of libraries but it has so much going on that many developers find it too complex. Tornado is a much simpler async framework to adapt to and allows you to run twisted libraries.

Twisted's inlinecallback's and tornado's gen module gets rid of all the async spaghetti code. This is hard to do with nodejs but I still chose nodejs because the available libraries made my project quicker to develop.

Re: Async I/O for Python 3

#45

Post by Demiurge »

benatkin wrote: post #25

Why does Guido think this is general purpose enough to add to Python but that the scientific features to make it competitive with R aren't? Is he envious of node.js?

I think because scientific features are not fundamental tools of expression, while he is working a language that is trying to be the foundation (most general) for the more specific libraries or tools.

Re: Async I/O for Python 3

#46

Post by RyanGWU82 »

hosay123 wrote: post #12
rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

Sane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran,…

My company used gevent in production at very large scale, and we were extremely happy with it. In fact, we ported our existing Django and Flask applications to run under gevent, which was a surprisingly fast process. (Weeks, not months, to port rather large codebases.) We did have to be careful with third-party libraries, like Zookeeper clients, but that was worth the tradeoff. We got the performance of an evented structure without having to rewrite a ton of code.

Re: Async I/O for Python 3

#47

Post by ekimekim »

A minor gripe (seperate from all my other gripes, which other people have already talked about):

"...run code in another thread - sometimes there is no alternative - eg. getaddrinfo(), database connections"

Just thought I'd mention that async-supporting DNS libs do exist (eg. gevent ships with C-ares), and in particular I've used async postgres database connections in both C and gevent. The code to gevent-ise psycopg2 connections is about 10 or 15 lines, iirc.

Re: Async I/O for Python 3

#49

Post by ekimekim »

stuaxo wrote: post #21
rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

Eh, the whole point is that this is at a lower level to all of those. All the above frameworks will be ported to this and become interoperable - in a similar way to how wsgi works for the web.

It won't be useful if the standard interface is, basically, Twisted.

Stack-based solutions like gevent, eventlet get screwed over by this.

Re: Async I/O for Python 3

#50

Post by ekimekim »

CoffeeDregs wrote: post #15

Perhaps the video makes more clear the rationale. E.g. Possible solution: "Standardizing gevent solves all its problems". One of the responses: "I like to write clean code from scratch". Another: "I really like clean interfaces". So I'd prefer that the BDFL work with the gevent folks to get it cleaned up and integrated while adjusting it to expose a "clean interface". Perhaps the whole thing will make more sense once…

I find that unlikely: Guido doesn't like gevent.

Though yes, it's a nice thought.

Post reply on HN