Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Origins of Python's “Functional” Features (2009)

python-history.blogspot.com

Post reply on HN
Print view View original

21–30 of 37 posts

Re: Origins of Python's “Functional” Features (2009)

#21

Post by maweki »

tommikaikkonen wrote: post #16

While it's far from a functional language, I find that it's possible to build functional style abstractions without too much headache. The iterator protocol is so well supported by the language and the standard libraries that using map, filter, itertools, generator expressions & comprehensions gets you a long way. You get the rest of the way by writing any specialized combinators you need and then imperative code to…

The state of a generator is always a problem that trips me up the most when I try to do FP-Python. In general a function shouldn't care whether the input was a list, set or generator. While iterating over it (or yielding from it), we always mutate some internal state of the iterator.

One solution would be, I think, to always pass tee-d copies of iterators and never(!) any iterator like it is (because you don't know whether next() has side effects or not). The other solution would be to not ever do anything lazy. One could also always consume all iterators fully. That way any generator re-use would be apparent immediately.

Some function that moves a generator forward just one or two items (something with the semantics of findFirst, for example), can subtly introduce bugs.

Re: Origins of Python's “Functional” Features (2009)

#22

Post by EvilTerran »

clusmore wrote: post #17
m_mueller wrote: post #14

Earlier quoted context omitted.

I guess my next question is then - why is anyone using filter/map instead of comprehensions or even generator expressions? Familiarity when coming from FP?

Well, for one I believe they pre-date comprehensions. Having them as functions is also occasionally useful for partial application, e.g. from functools import partial to_strings = partial(map, str) # vs def to_strings(seq): return (str(elem) for elem in seq)

Or this:

    def to_strings(seq):
        return map(str, seq)
Generally, when the operation I'm applying to each element happens to already be a named function, I find "map(f, seq)" preferable to "(f(x) for x in seq)".

Re: Origins of Python's “Functional” Features (2009)

#23

Post by pmontra »

m_mueller wrote: post #14
viraptor wrote: post #13

Earlier quoted context omitted.

~4.5 times faster with comprehension. In [11]: timeit.timeit('''list(filter(lambda x : ('widgets' in x), mixed_widgets))[0]['widgets']''', '''nw={'abc': 'def'};w={'widgets':'s'};mixed_widgets=[nw]*100+[w]+[nw]*100''', number=100000) Out[11]: 2.6956532129988773 In [12]: timeit.timeit('''[x for x in mixed_widgets if 'widgets' in x][0]['widgets']''', '''nw={'abc': 'def'};w={'widgets':'s'};mixed_widgets=[nw]*100+[w]+[nw]…

I guess my next question is then - why is anyone using filter/map instead of comprehensions or even generator expressions? Familiarity when coming from FP?

Coming from Ruby it's easier to use map, because it's what Ruby's standard library offers.

However comprehensions are not that harder when one finally decides to understand how they work. Not as readable as map() IMHO. Example:

Ruby

    [1, 2, 3].map {|x| x*x} # object.method(args)
vs Python

    [x*x for x in [1, 2, 3]]
where we have the function first, then the definition of the variable, then the data. This is the opposite of the object.method OO notation and using a variable before defining it is not what we usually do. But it's almost the usual mathematical notation "for i in set do f(i)" with the function at the beginning.

Not a big deal.

About a problem raised in a comment of the post (which is from 2009): this is Guido (2009) about the lack of tail call optimization in Python http://neopythonic.blogspot.it/2009/04/tail-recursion-elimin...

Re: Origins of Python's “Functional” Features (2009)

#24

Post by xapata »

maweki wrote: post #21
tommikaikkonen wrote: post #16

While it's far from a functional language, I find that it's possible to build functional style abstractions without too much headache. The iterator protocol is so well supported by the language and the standard libraries that using map, filter, itertools, generator expressions & comprehensions gets you a long way. You get the rest of the way by writing any specialized combinators you need and then imperative code to…

The state of a generator is always a problem that trips me up the most when I try to do FP-Python. In general a function shouldn't care whether the input was a list, set or generator. While iterating over it (or yielding from it), we always mutate some internal state of the iterator. One solution would be, I think, to always pass tee-d copies of iterators and never(!) any iterator like it is (because you don't know w…

Make your functions consumers. Don't pass tee'd iterators in, but tee after receiving if necessary.

I don't see how find_first would cause any more problems than filter or sum.

Re: Origins of Python's “Functional” Features (2009)

#25

Post by mangecoeur »

I have sometimes wondered why python used 'lambda' for anonymous functions rather than just... well literally anonymous function - normal function syntax without the name. Could have been something like

`def (x,y): x+y`

This makes it sound like that was just how it was implemented. Nothing fundamentally wrong with lambda i guess

Re: Origins of Python's “Functional” Features (2009)

#26

Post by wheaties »

I never understood his desire to remove 'reduce' from Python. I would love to hear a cogent reason as to why we should have to write a for-loop for everything 'reduce' like. In other languages, 'reduce' (akin to 'fold' for non-Python people) is one of the things you reach for daily.

Re: Origins of Python's “Functional” Features (2009)

#27

Post by Tulip68 »

xapata wrote: post #24
maweki wrote: post #21

Earlier quoted context omitted.

The state of a generator is always a problem that trips me up the most when I try to do FP-Python. In general a function shouldn't care whether the input was a list, set or generator. While iterating over it (or yielding from it), we always mutate some internal state of the iterator. One solution would be, I think, to always pass tee-d copies of iterators and never(!) any iterator like it is (because you don't know w…

Make your functions consumers. Don't pass tee'd iterators in, but tee after receiving if necessary. I don't see how find_first would cause any more problems than filter or sum.

That is a very bad idea in general and will lead to enormous memory leaks when large tee'd iterators get out of sync with one another.

See here:

This itertool may require significant auxiliary storage (depending on how much temporary data needs to be stored). In general, if one iterator uses most or all of the data before another iterator starts, it is faster to use list() instead of tee().

https://docs.python.org/2/library/itertools.html#itertools.t...

Re: Origins of Python's “Functional” Features (2009)

#28

Post by empath75 »

wheaties wrote: post #26

I never understood his desire to remove 'reduce' from Python. I would love to hear a cogent reason as to why we should have to write a for-loop for everything 'reduce' like. In other languages, 'reduce' (akin to 'fold' for non-Python people) is one of the things you reach for daily.

`from functools import reduce`

Re: Origins of Python's “Functional” Features (2009)

#29

Post by d0mine »

wheaties wrote: post #26

I never understood his desire to remove 'reduce' from Python. I would love to hear a cogent reason as to why we should have to write a for-loop for everything 'reduce' like. In other languages, 'reduce' (akin to 'fold' for non-Python people) is one of the things you reach for daily.

It is still available as functools.reduce

An explicit loop is easier to read as a rule than the corresponding reduce() code. See http://stackoverflow.com/questions/15995/useful-code-which-u...

Re: Origins of Python's “Functional” Features (2009)

#30

Post by jghn »

d0mine wrote: post #29
wheaties wrote: post #26

I never understood his desire to remove 'reduce' from Python. I would love to hear a cogent reason as to why we should have to write a for-loop for everything 'reduce' like. In other languages, 'reduce' (akin to 'fold' for non-Python people) is one of the things you reach for daily.

It is still available as functools.reduce An explicit loop is easier to read as a rule than the corresponding reduce() code. See http://stackoverflow.com/questions/15995/useful-code-which-u...

"Easier to read" for some people, I think you mean
Post reply on HN