What Color Is Your Function? • Hacker News Board

Re: What Color Is Your Function?

#151

Post by Guvante » Mon, Feb 09, 2015, 10:32 PM UTC

rco8786 wrote: post #81

Guvante wrote: post #76

Earlier quoted context omitted.

Go seems to have solved the IO problem, but what about the general asynchronous issue? How do you handle calling a function several thousand times on dozens of threads and combine the results?

Is there any reason you couldn't just spin up so goroutines to do this?

Telling a hundred threads to do work is easy, figuring out how to sync all of the answers is the hard part.

I am not saying you can't pull it off, but the proposed solution doesn't simplify the complex part of those operations.

Re: What Color Is Your Function?

#152

Post by glyph » Wed, Feb 11, 2015, 12:45 AM UTC

munificent wrote: post #121

kmike84 wrote: post #109

https://glyph.twistedmatrix.com/2014/02/unyielding.html is a good read - there are reasons why explicit sync-async "coloring" (i.e. await/yield) is better than green threads/coroutines which author admires.

That's a really long post, and I dimly recall reading it a while back, but after you scrape off it saying the same thing over and over again, I think it reduces down to an uncompelling argument. It's basically, "I want context switches syntactically explicit in my code. If they aren't, reasoning about it is exponentially harder." And I think that's pretty clearly a strawman. Everything the author claims about threade…

I don't think you read the article very carefully. Specifically, what I am claiming about (shared-state) threaded code which is not true of "any re-entrant code" is the fact that you cannot tell whether threaded code is re-entrant or not without a comprehensive, combinatorial whole-program analysis. It's not feasible to know what you might be re-entrant with, because you might be re-entrant with anything. With preemptive threads you really just can't do it at all, but with green threads you have to follow every call stack all the way to the bottom, because nothing else about it the bottom of the stack tells you whether you're going to context switch or not. When you get back a Deferred (or a Promise or a Future or a thunk or whatever), then the stack-inspection can be shallow; O(1) on the depth of your stack instead of O((stack_depth) * (function_length)).

Re: What Color Is Your Function?

#153

Post by cben » Thu, Feb 12, 2015, 2:23 PM UTC

munificent wrote: post #49

andybak wrote: post #28

Earlier quoted context omitted.

Funny to consider this alongside Guido's refusal to add full anonymous functions to Python. His argument seems to be "If it's too long for a single-line lambda, then it's long enough to deserve a name"

Guido has a different, legitimate reason to not add multi-line lambdas to Python: they are super nasty to integrate with Python's grammar. Python has a strict grammar where statements (which use indentation) contain expressions, but never vice versa. Allowing statement-body lambdas would give you an expression form that contains significant indentation that could be embedded in the middle of some larger expression, l…

Python also has `yield` generator syntax which can approximate `async` syntax rather well (everything is still colored but less painful). There seems to be no concensus whether generators or promises ("futures") are better but it seems to be agreed that if you want nice syntax you should be happy with generators.

The worst problem is stdlib APIs weren't designed for any async style. => gevent's monkey-patching all of stdlib is considered among the _less_ nasty solutions :-(

On a deeper level, I don't think the Python community even agrees async is a very important problem. Unlike the self-selection Node enjoyed, many people work on software where mostly sync code is good enough.