Counting Things in Python: A History • Hacker News Board

Re: Counting Things in Python: A History

#31

Post by dalke » Tue, Nov 10, 2015, 6:50 AM UTC

pvg wrote: post #23

dalke wrote: post #22

Earlier quoted context omitted.

That's not a change to the iterator protocol. It's a change to 'StopIteration handling inside generators'. The following will still raise a StopIteration in Python 3.5+: >>> next(iter([]))

Yep, the fundamental termination mechanism seems to be the same. Years and years ago I got into some silly irc nerdgument with a python expert (I think one of the twisted people) about the ugliness of this design and for a moment I thought I got to triumphantly yell 'Told you so!' a decade later. Alas, not the case.

I take it you prefer the explicit test for the end of iteration?

As an historic note, the StopIteration form grew out of the earlier iterator form, which called __getitem__ with successive integers until there was an IndexError. That may explain a bias towards an exception-based approach.

Re: Counting Things in Python: A History

#32

Post by vog » Tue, Nov 10, 2015, 7:02 AM UTC

The author seems to misunderstand one part of The Zen of Python:

| Simple is better than complex.

by saying:

> Our code is more complex (O(n2) instead of O(n)), less beautiful, and less readable

The Zen is not about computation complexity! It's about complexity of the source code.

The code in question is:

    color_counts = dict((c, colors.count(c)) for c in set(colors))

And while I agree that this is inefficient and shouldn't be used in a library, I find this to be very readable and would always prefer that code if I know that I'm dealing only with small lists. It translates neatly to a natural-language description of the problem:

    Give me a dictionary that maps each distinct color
    to the number of times it occurs in the list.

So I don't think this violates the guide "Simple is better than complex". Rather, it is a good example where it makes sense to introduce additional complexity to improve the performance of an often-used helper function.

Re: Counting Things in Python: A History

#33

Post by raymondh » Tue, Nov 10, 2015, 7:05 AM UTC

mgraczyk wrote: post #8

Worth noting that Counter itself uses what the article calls get Method , but with a common performance optimization (caching a bound method). def _count_elements(mapping, iterable): mapping_get = mapping.get for elem in iterable: mapping[elem] = mapping_get(elem, 0) + 1

Also worth noting that those lines are immediately followed by:

    try:                                    # Load C helper function if available        
        from _collections import _count_elements
    except ImportError:
        pass

Re: Counting Things in Python: A History

#34

Post by jboggan » Tue, Nov 10, 2015, 7:40 AM UTC

buzzdenver wrote: post #30

This makes me appreciate autovivification and casting in perl so that you can just say "$color_counts{$color} += 1" without all the initialization.

Yes! This was one of the biggest things I missed when I moved to Ruby. I try and tell people how great autovivification is but unless they've coded with it the feature just sounds strange. But it lets you build some really great data structures on the fly!

Re: Counting Things in Python: A History

#35

Post by esfandia » Tue, Nov 10, 2015, 7:51 AM UTC

Would be interesting doing the same exercise for Java as well. We've been through Hashtable, HashMap, basic for loops, Enumeration, Iterator, foreach loops, generics, and now we have lambdas.

Re: Counting Things in Python: A History

#36

Post by kbenson » Tue, Nov 10, 2015, 7:55 AM UTC

jboggan wrote: post #34

buzzdenver wrote: post #30

This makes me appreciate autovivification and casting in perl so that you can just say "$color_counts{$color} += 1" without all the initialization.

Yes! This was one of the biggest things I missed when I moved to Ruby. I try and tell people how great autovivification is but unless they've coded with it the feature just sounds strange. But it lets you build some really great data structures on the fly!

I love it as well, and would hate to do without it, but it does come with it's own warts. Such as this:

    use Data::Dumper;
    my %h;
    if ( $h{foo}{bar}{baz} ) { say "Never happens"; }
    say Dumper \%h;

And you get this:

    $VAR1 = {
              'foo' => {
                         'bar' => {}
                       }
            };

Re: Counting Things in Python: A History

#37

Post by dec0dedab0de » Tue, Nov 10, 2015, 8:26 AM UTC

I thought this was going to end around 2.4 with the .count(c) method, since you don't really need the dict at that point.

Re: Counting Things in Python: A History

#38

Post by adrianN » Tue, Nov 10, 2015, 8:47 AM UTC

vog wrote: post #32

The author seems to misunderstand one part of The Zen of Python: | Simple is better than complex. by saying: > Our code is more complex (O(n2) instead of O(n)), less beautiful, and less readable The Zen is not about computation complexity! It's about complexity of the source code. The code in question is: color_counts = dict((c, colors.count(c)) for c in set(colors)) And while I agree that this is inefficient and sho…

I don't think you should get into the habit of writing O(n^2) algorithms if the O(n) solution is not much more complex. Unless the constants are very dissimilar, you probably hurt your performance already for a few hundred elements. Writing reusable code includes using algorithms that are ok for a wide range of input sizes.

Re: Counting Things in Python: A History

#39

Post by pvg » Tue, Nov 10, 2015, 9:21 AM UTC

dalke wrote: post #31

pvg wrote: post #23

Earlier quoted context omitted.

Yep, the fundamental termination mechanism seems to be the same. Years and years ago I got into some silly irc nerdgument with a python expert (I think one of the twisted people) about the ugliness of this design and for a moment I thought I got to triumphantly yell 'Told you so!' a decade later. Alas, not the case.

I take it you prefer the explicit test for the end of iteration? As an historic note, the StopIteration form grew out of the earlier iterator form, which called __getitem__ with successive integers until there was an IndexError. That may explain a bias towards an exception-based approach.

I probably do, although in practice, given how well (and composable) python comprehensions/iterators/generators have turned out, getting all worked up about some implementation detail now seems a bit churlish and pointless.

Re: Counting Things in Python: A History

#40

Post by reikonomusha » Tue, Nov 10, 2015, 9:25 AM UTC

jlarocco wrote: post #16

kazinator wrote: post #11

$ txr This is the TXR Lisp interactive listener of TXR 123. Use the :quit command or type Ctrl-D on empty line to exit. 1> [hash-update [group-by identity '(brown red green yellow yellow brown brown black)] length] #H(() (green 1) (red 1) (brown 3) (black 1) (yellow 2)) Form a hash by grouping like items into lists. The identity function is the key in the hash and the basis for equality, so the keys are colors, and t…

I guess this is off topic, but neat language. But that algorithm allocates a bunch of intermediate lists and iterates through the hash table when it doesn't need to. Here it is in common lisp: (defun count-elements (lst) (loop with rval = (make-hash-table) for val in lst do (incf (gethash val rval 0)) finally (return rval)))

Pedantry: this won't work with strings if they're not EQL.