Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async I/O for Python 3

dropbox.com

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Async I/O for Python 3

#61

Post by jessaustin »

simonw wrote: post #40
rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

If you're using monkey patching in the standard library for a language, something has gone very wrong.

This is really a matter of taste. You should at least be aware that you're monkeypatching, and code and test accordingly. Many people have good results from monkeypatching, and even more have good results from calling well-written-and-tested libraries that monkeypatch.

Re: Async I/O for Python 3

#62

Post by antihero »

RyanGWU82 wrote: post #46
hosay123 wrote: post #12

Earlier quoted context omitted.

Sane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran,…

My company used gevent in production at very large scale, and we were extremely happy with it. In fact, we ported our existing Django and Flask applications to run under gevent, which was a surprisingly fast process. (Weeks, not months, to port rather large codebases.) We did have to be careful with third-party libraries, like Zookeeper clients, but that was worth the tradeoff. We got the performance of an evented st…

How does one go about porting Django apps to be compatible with gevent? In that you used gevent in your Django code, or that you built something completely different?

Re: Async I/O for Python 3

#63

Post by antihero »

RyanGWU82 wrote: post #46
hosay123 wrote: post #12

Earlier quoted context omitted.

Sane is in the eye of the beholder.. gevent looks nice, but I'd be very diffident when it comes to actually supporting it in production. It monkey patches the standard library and messes with CPython internals to achieve what it does, infinitely increasing the chance it will conflict with some other piece of code (for example, that bizarre ancient internal propriety library you're using that started life in Fortran,…

My company used gevent in production at very large scale, and we were extremely happy with it. In fact, we ported our existing Django and Flask applications to run under gevent, which was a surprisingly fast process. (Weeks, not months, to port rather large codebases.) We did have to be careful with third-party libraries, like Zookeeper clients, but that was worth the tradeoff. We got the performance of an evented st…

How does one go about porting Django apps to be compatible with gevent? In that you used gevent in your Django code, or that you built something completely different?

Re: Async I/O for Python 3

#64

Post by cpprototypes »

rdtsc wrote: post #54
nashequilibrium wrote: post #44

Earlier quoted context omitted.

The reason why you getting so many downvotes is because your comment is not just silly but flat wrong. Twisted, Tornado were around before nodejs. There are also async frameworks that make writing async code the same as synchronous code. I like tornado but I am using nodejs for my current app because of the libraries. This is where nodejs really shines, the community and libraries are awesome. Twisted has a lot of li…

Sorry I didn't express myself correctly (see my reply to akuchling below). Basically yes, Python had Twisted for years, it had Diesel, Monocle, Tornado, and some other ones. I am aware of those and as you've read my comment you saw that I used Twisted enough to know its ins and outs (5 years). > There are also async frameworks that make writing async code the same as synchronous code. Yes there is inlineCallbacks and…

I have a question for you since you have a lot of experience with async. Eventually node.js will have generators (when V8 implements ECMAScript 6) which should allow node.js to have something like gevent. What kind of effect do you think this will have on the node.js world?

Re: Async I/O for Python 3

#65

Post by ufo »

johnsoft wrote: post #57
Locke1689 wrote: post #33

As Guido mentions, @coroutine/yield from is very similar to C#'s async implementation (with some differences like type safety). Since Guido has the barest of descriptions on how this works, you may find the C# async description useful. [1] [1] http://msdn.microsoft.com/en-us/library/vstudio/hh191443.asp...

Just to check if I'm understanding the presentation right, will the implementation involve compiler magic to turn this: @coroutine def getresp(): s = socket() yield from loop.sock_connect(s, host, port) yield from loop.sock_sendall(s, b'xyzzy') data = yield from loop.sock_recv(s, 100) # ... into this, similar to how C# does it? (let's pretend multi-line lambdas exist for a minute) def getresp(): s = socket() loop.soc…

I don't understand your question. From the implementation perspective Python doesn't rewrite things to continuation-passing-style but the end result should be the same.

Re: Async I/O for Python 3

#66

Post by pjscott »

antihero wrote: post #62
RyanGWU82 wrote: post #46

Earlier quoted context omitted.

My company used gevent in production at very large scale, and we were extremely happy with it. In fact, we ported our existing Django and Flask applications to run under gevent, which was a surprisingly fast process. (Weeks, not months, to port rather large codebases.) We did have to be careful with third-party libraries, like Zookeeper clients, but that was worth the tradeoff. We got the performance of an evented st…

How does one go about porting Django apps to be compatible with gevent? In that you used gevent in your Django code, or that you built something completely different?

I assume that you use a WSGI server which allocates one greenlet per request (e.g. gevent.wsgi, or Gunicorn's async workers), and make sure that the rest of your code isn't going to block the event loop too much. Once that's done, you can have a whole bunch of HTTP requests being handled at once. That's nice if your server spends most of its time blocking on database requests or something.

Re: Async I/O for Python 3

#67

Post by stock_toaster »

jlgreco wrote: post #59
ceronman wrote: post #48

Earlier quoted context omitted.

Twisted with inlineCallbacks is actually quite nice IMO. It's very similar to C# async/await.

Twisted's inlineCallbacks singlehandedly turn Twisted in my mind from an abomination into something that is a joy to work with. In lei of a more hands off Erlang/Go approach, I am convinced that style is the only way to go.

inlineCallbacks made twisted palatable. It didn't make it good.

Re: Async I/O for Python 3

#68

Post by codygman »

jessaustin wrote: post #61
simonw wrote: post #40

Earlier quoted context omitted.

If you're using monkey patching in the standard library for a language, something has gone very wrong.

This is really a matter of taste. You should at least be aware that you're monkeypatching, and code and test accordingly. Many people have good results from monkeypatching, and even more have good results from calling well-written-and-tested libraries that monkeypatch.

But monkey patching turns python from explicit to implicit. It just doesn't feel pythonic to me, and I don't think I'm alone in this.

A big reason I use (and enjoy using) python is because it doesn't feel like a "bolted on" solution. All current concurrency options for python feel bolted on to me personally.

The python internals weren't designed for this, which is the reason they have to use monkey patching. It doesn't mean that you can't make something work, but it means the language/interpreter sure aren't going to help you make it work.

I just couldn't write something which depends on speed and concurrency in python right now, knowing there are solutions much better designed for the problem. Python holds a special place in my heart, but unicode and concurrency aren't so good right now.

However, this future async support and unicode support in python 3 very much excites me!

Re: Async I/O for Python 3

#69

Post by aba_sababa »

rdtsc wrote: post #10

Sigh, another Async framework. Yes it has nice features such can replace the reactor/hub thing. Has futures/promises/deferreds. That has all been done before in Twisted. Yields are cute and there was monocle, I wouldn't say it exactly took off : https://github.com/saucelabs/monocle Twisted has inlineCallbacks that use yields as well. Just import Twisted into stdlib then and use that. I am surprised that gevent was di…

The only reason I'm not doing that is because I'm anchored by Python's stats libraries.
Post reply on HN