Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Python yield keyword explained

stackoverflow.com

Post reply on HN
Print view View original

31–40 of 53 posts

Re: The Python yield keyword explained

#31

Post by gejjaxxita »

jiggy2011 wrote: post #30

What I find weird about the yield keyword is that it seems to effect the execution of code that comes before the statement, causing it not to execute until later. For example: def createGenerator(): print "aaa" mygen = createGenerator() outputs: aaa Whereas def createGenerator(): print "aaa" yield mygen = createGenerator() Outputs nothing.

The presence of the yield keyword in a function definition means the function will return a generator when it's called. Therefore "print "aaa"" is not executed on "mygen = createGenerator()" as expected. "print "aaa"" is infact executed when mygen.next() is called.

To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute any of the code. This is why the following results in an error:

  >>> def hello(j):
  ...     if j>2:
  ...         return "big number"
  ...     elif j
A return keyword cannot exist within the body of a generator - it makes no sense.

A more detailed explanation can be found here http://docs.python.org/2.5/ref/yieldexpr.html

Re: The Python yield keyword explained

#32

Post by Leszek »

to3m wrote: post #16
johnsonqq wrote: post #11

Earlier quoted context omitted.

It's precisely all the help you need to go and look up your notes on coroutines from college. You did go to college didn't you? Or are you an anti-college hipster, suddenly finding the limitations of that approach.

I studied computer science at a university, not just a college, and coroutines weren't covered during the course. So even if you've got a degree, there's no guarantee you'll have met a coroutine! (I do agree that a formal education is valuable, even if you already think (or know) you're good at programming. I learned a lot from my course. There's a lot to be said for being forced to learn a bunch of stuff, and when y…

> at a university, not just a college

College is (most of the time) American for university.

Re: The Python yield keyword explained

#33

Post by Leszek »

johnsonqq wrote: post #10
kmfrk wrote: post #8

Earlier quoted context omitted.

Does studying CS at university make people as insufferably grumpy and unhelpful as you?

Come on, I've got a reasonable point here. People post on forums asking these questions which should have been answered during their educations. If civil engineers had a forum site and someone asked "how do I calculate the load a simple beam", people would say, what the fuck are you doing in your job if you don't know that? Who is employing you and why? Where did you go to college? Who was your tutor because next tim…

One of the false assumptions you're making here is that CS == programming. There's a difference between being a civil engineer, where you have to be able to calculate loads on beams, and being a physicist, where you try to understand why beams behave the way they do.

Re: The Python yield keyword explained

#34

Post by jiggy2011 »

gejjaxxita wrote: post #31
jiggy2011 wrote: post #30

What I find weird about the yield keyword is that it seems to effect the execution of code that comes before the statement, causing it not to execute until later. For example: def createGenerator(): print "aaa" mygen = createGenerator() outputs: aaa Whereas def createGenerator(): print "aaa" yield mygen = createGenerator() Outputs nothing.

The presence of the yield keyword in a function definition means the function will return a generator when it's called. Therefore "print "aaa"" is not executed on "mygen = createGenerator()" as expected. "print "aaa"" is infact executed when mygen.next() is called. To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute…

I think I mostly understand the idea, the presence of the yield statement transforms the nature of the parent function itself and makes evaluation lazy.

This seems a little weird syntactically though because it feels like the interpreter is somehow reading ahead in the program. It also means you don't necessarily know that this is a lazy function unless you read to the end.

A more intuitive syntax might be something like:

  def lazy createGenerator():
    print "aaa"
    yield "bbb"

Re: The Python yield keyword explained

#35

Post by jlgreco »

stch2 wrote: post #27
phpnode wrote: post #25

Earlier quoted context omitted.

(replying to myself because I can't reply to stch directly): What does that mean/how does it affect me? It means that only people who have specifically turned "showdead" on can see your posts. No one can reply to you posts directly. Your upvotes do nothing and don't count. But crucially, to you, everything seems completely normal. And howd that happen? An administrator decided that your first ever comment didn't fit…

Wow, that's pretty ridiculous. You'd figure after a few months they'd tell me, right? Is that also why I get "Unknown or expired link" all the time and every page always takes 15-20 seconds to load? I noticed when I logged out that issue went away. I always just assumed the forum was really poorly designed. :o)

> Is that also why I get "Unknown or expired link"

No, that is just something HN does.

Re: The Python yield keyword explained

#36

Post by ars »

A helpful hint on the name: Yield should be read like "Yields" as in "returns" or "produces".

Not Yield as in the common multithreading command to allow another thread to run.

PS. Anyone know why they chose such a confusing name?

Re: The Python yield keyword explained

#37

Post by csense »

jiggy2011 wrote: post #34
gejjaxxita wrote: post #31

Earlier quoted context omitted.

The presence of the yield keyword in a function definition means the function will return a generator when it's called. Therefore "print "aaa"" is not executed on "mygen = createGenerator()" as expected. "print "aaa"" is infact executed when mygen.next() is called. To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute…

I think I mostly understand the idea, the presence of the yield statement transforms the nature of the parent function itself and makes evaluation lazy. This seems a little weird syntactically though because it feels like the interpreter is somehow reading ahead in the program. It also means you don't necessarily know that this is a lazy function unless you read to the end. A more intuitive syntax might be something…

This syntax would against the Python principle DRY (Don't Repeat Yourself): The "lazy" keyword conveys no information that doesn't exist in the function body.

Also, Python has precedent for this scanning behavior. One of the fundamental features of Python is that the scope of a variable is determined by the presence of an assignment statement in a function. For example:

  a = 1
  def f():
     print a      # UnboundLocalError occurs here
     a = 2
  f()
If you know Javascript, think about how often you need to use the "var" keyword in that language, and how easy it is to accidentally pollute the global namespace by omitting it.

Re: The Python yield keyword explained

#38

Post by csense »

Yttrill wrote: post #29
bthomas wrote: post #5

Earlier quoted context omitted.

I use yield and generators regularly. They just make sense, but I never knew what a coroutine is in an academic sense

This does reflect very sadly on schools, and shows up when trying to explain things to programmers used to conventional languages, especially if you're trying to explain why C is fundamentally broken. Roughly there are three fundamental control transfer operations: conditional jumps, subroutine calls, and coroutine calls. A subroutine is a slave, the caller is a master. With coroutines, the caller and callee are peer…

> Coroutine calling is more fundamental and easier to program

Citation needed.

For coroutines, each coroutine needs its own stack. That means you have to have a dynamic memory system baked into the language. And maybe garbage collection too.

> on today's badly designed CPU's you have to think of coroutines as requiring stack swapping

I can't think of an implementation of yield (let alone general coroutines) that doesn't require a separate stack for each coroutine. I admit that I haven't learned very many of the stranger forgotten architectures that are out there, so I might be blinded by the limitations of a somewhat conventional experience.

But I'm also thinking it might even be provable that each coroutine needs its own stack: Think of a program that has m generator functions, where f_1 calls f_2, f_2 calls f_3, ..., f_{m-1} calls f_m. Each of these subroutines creates n copies of its next-level generator, and steps those generators and yields to the parent unpredictably (for example, depending on input from a user-supplied file). It seems like if m and n are large enough, you'll have no choice but to resort to swapping stacks.

Re: The Python yield keyword explained

#39

Post by Someone »

ars wrote: post #36

A helpful hint on the name: Yield should be read like "Yields" as in "returns" or "produces". Not Yield as in the common multithreading command to allow another thread to run. PS. Anyone know why they chose such a confusing name?

Because it allows another thread to run. Yes, they actually are coroutines, but that is an implementation detail. Just look at it as an optimization for the case where at most one of the two threads (the one producing the values and the one consuming it) runs at a time.

Re: The Python yield keyword explained

#40

Post by andreif »

jiggy2011 wrote: post #30

What I find weird about the yield keyword is that it seems to effect the execution of code that comes before the statement, causing it not to execute until later. For example: def createGenerator(): print "aaa" mygen = createGenerator() outputs: aaa Whereas def createGenerator(): print "aaa" yield mygen = createGenerator() Outputs nothing.

The answer on SO says explicitly (in bold, BTW):

> To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is bit tricky :-)

Post reply on HN