Great blog post Trey!
Counting Things in Python: A History
51–60 of 61 posts
Re: Counting Things in Python: A History
#52$ txr This is the TXR Lisp interactive listener of TXR 123. Use the :quit command or type Ctrl-D on empty line to exit. 1> [hash-update [group-by identity '(brown red green yellow yellow brown brown black)] length] #H(() (green 1) (red 1) (brown 3) (black 1) (yellow 2)) Form a hash by grouping like items into lists. The identity function is the key in the hash and the basis for equality, so the keys are colors, and t…
I guess this is off topic, but neat language. But that algorithm allocates a bunch of intermediate lists and iterates through the hash table when it doesn't need to. Here it is in common lisp: (defun count-elements (lst) (loop with rval = (make-hash-table) for val in lst do (incf (gethash val rval 0)) finally (return rval)))
I've come up with a function group-reduce which generalizes the construction of hashes from sequences, while streamlining the syntax. The idea is that the hash table entries are accumulators, and we can combine the grouping operation which distributes items into hash keys with reduction over the accumulators. For instance, take the integers 1 through 10, group them into even and odd, and sum the groups:
1> [group-reduce (hash) evenp + (range 1 10) 0]
#H(() (t 30) (nil 25))
Now with this, we can obtain a histogram easily, because a left (or right) reduce/fold can count items:
2> [reduce-left (do inc @1)
'(brown red green yellow yellow brown brown black) 0]
8
3> [group-reduce (hash) identity (do inc @1)
'(brown red green yellow yellow brown brown black) 0]
#H(() (black 1) (yellow 2) (green 1) (red 1) (brown 3))
I deliberately made it accept an existing hash so it can be called repeatedly on the same hash to accumulate multiple jobs. That also answers the question of where to specify the hash table attributes.
One last detail is that there is an optional argument we are omitting in the group-by call: a filter function which is optionally applied to each element in the table after the accumulation is done. For instance, we can specify this as nreverse, and then we can express group-by using group-reduce.
First naive attempt: oops cons arguments wrong way:
1> [group-reduce (hash) evenp cons (range 1 10) nil]
#H(() (t (((((nil . 2) . 4) . 6) . 8) . 10)) (nil (((((nil . 1) . 3) . 5) . 7) . 9)))
2> [group-reduce (hash) evenp [flipargs cons] (range 1 10) nil]
#H(() (t (10 8 6 4 2)) (nil (9 7 5 3 1)))
3> [group-reduce (hash) evenp [flipargs cons] (range 1 10) nil nreverse]
#H(() (t (2 4 6 8 10)) (nil (1 3 5 7 9)))
4> [group-by evenp (range 1 10)]
#H(() (t (2 4 6 8 10)) (nil (1 3 5 7 9)))Re: Counting Things in Python: A History
#53Earlier quoted context omitted.
I love it as well, and would hate to do without it, but it does come with it's own warts. Such as this: use Data::Dumper; my %h; if ( $h{foo}{bar}{baz} ) { say "Never happens"; } say Dumper \%h; And you get this: $VAR1 = { 'foo' => { 'bar' => {} } };
You'd want to use the exists operator in that case. It checks if the hash key is present without auto vivifying it. my %hash = (); if (exists $hash{foo}) {print "This doesn't run!";}
I'm well aware of exists, and how it functions, and that it in no way solves the problem presented. Exists tests for the existence of a hash key, so you can tell if it exists but is possibly undefined, but it does not stop autovivification in any way.
Your example doesn't cause autovivification even without exists. Autovivification is the automatic creation of the underlying hashes and arrays in a multiple level data structure when they are used while accessing a nested data-structure.
For example, given an empty hash %hash, $hash{foo} does not cause autovivification, but $hash{foo}{bar} will automatically create an empty hash and assign a reference to it to $hash{foo}.
Re: Counting Things in Python: A History
#54This makes me appreciate autovivification and casting in perl so that you can just say "$color_counts{$color} += 1" without all the initialization.
Asking as a Perl ignoramus, what does this give you over the defaultdict(int) example from the article?
Does autovivification mean that the default value to use depends on the operation? eg 0 for addition and "" for concatenation?
Re: Counting Things in Python: A History
#55Earlier quoted context omitted.
Yes! This was one of the biggest things I missed when I moved to Ruby. I try and tell people how great autovivification is but unless they've coded with it the feature just sounds strange. But it lets you build some really great data structures on the fly!
I love it as well, and would hate to do without it, but it does come with it's own warts. Such as this: use Data::Dumper; my %h; if ( $h{foo}{bar}{baz} ) { say "Never happens"; } say Dumper \%h; And you get this: $VAR1 = { 'foo' => { 'bar' => {} } };
Fwiw Perl 6 only autovivifies anything, including intermediary data structure levels, when writing to a data structure. So nothing happens in this case:
my %h;
dd %h
if %h { say "Never happens" }
dd %h
Hash $var = {}
Hash $var = {}Re: Counting Things in Python: A History
#56Earlier quoted context omitted.
I love it as well, and would hate to do without it, but it does come with it's own warts. Such as this: use Data::Dumper; my %h; if ( $h{foo}{bar}{baz} ) { say "Never happens"; } say Dumper \%h; And you get this: $VAR1 = { 'foo' => { 'bar' => {} } };
Fwiw Perl 6 only autovivifies anything, including intermediary data structure levels, when writing to a data structure. So nothing happens in this case: my %h; dd %h if %h { say "Never happens" } dd %h prints: Hash $var = {} Hash $var = {}
Nice, I wasn't aware of that feature, but the features I'm still not aware of in Perl 6 could fill a small book, since I still haven't gotten around to using it for much. :/
Re: Counting Things in Python: A History
#57This makes me appreciate autovivification and casting in perl so that you can just say "$color_counts{$color} += 1" without all the initialization.
Asking as a Perl ignoramus, what does this give you over the defaultdict(int) example from the article? Does autovivification mean that the default value to use depends on the operation? eg 0 for addition and "" for concatenation?
I'm a Python ignoramus, but I think they're the same, except for Perl's being more concise. How would you do a 3-level hash though in Python ?
What you call "value to use depends on the operation" is called casting. Depending on the context, a variable can be different values. For example a variable that has been defined but nothing else is treated as 0 if you try to add a number to it. It's an empty string if you use it in a string operation.
Wikipedia explains it well: autovivification is the automatic creation of new arrays and hashes as required every time an undefined value is dereferenced. Perl autovivification allows a programmer to refer to a structured variable, and arbitrary sub-elements of that structured variable, without expressly declaring the existence of the variable and its complete structure beforehand.
Re: Counting Things in Python: A History
#58Earlier quoted context omitted.
Asking as a Perl ignoramus, what does this give you over the defaultdict(int) example from the article? Does autovivification mean that the default value to use depends on the operation? eg 0 for addition and "" for concatenation?
I'm a Python ignoramus, but I think they're the same, except for Perl's being more concise. How would you do a 3-level hash though in Python ? What you call "value to use depends on the operation" is called casting. Depending on the context, a variable can be different values. For example a variable that has been defined but nothing else is treated as 0 if you try to add a number to it. It's an empty string if you us…
> I'm a Python ignoramus, but I think they're the same, except for Perl's being more concise.
Isn't everything always more concise in Perl? No matter what language it's compared to :)
> How would you do a 3-level hash though in Python ?
You might/should be able to wrangle that with some customisation of the default_factory or a defaultdict subclass, but yeah the arbitrary depth wouldn't be as 'plug n play' as with Perl.
> What you call "value to use depends on the operation" is called casting. Depending on the context, a variable can be different values. For example a variable that has been defined but nothing else is treated as 0 if you try to add a number to it. It's an empty string if you use it in a string operation.
Yeah Python's strong typing (usually) balks at automatic casting. You'd need to choose what the default is.
Re: Counting Things in Python: A History
#59$ txr This is the TXR Lisp interactive listener of TXR 123. Use the :quit command or type Ctrl-D on empty line to exit. 1> [hash-update [group-by identity '(brown red green yellow yellow brown brown black)] length] #H(() (green 1) (red 1) (brown 3) (black 1) (yellow 2)) Form a hash by grouping like items into lists. The identity function is the key in the hash and the basis for equality, so the keys are colors, and t…
One way to do it in Perl 6:
> .Mix
mix(red, yellow(2), brown(3), black, green)
Re: Counting Things in Python: A History
#60Earlier quoted context omitted.
I'm a Python ignoramus, but I think they're the same, except for Perl's being more concise. How would you do a 3-level hash though in Python ? What you call "value to use depends on the operation" is called casting. Depending on the context, a variable can be different values. For example a variable that has been defined but nothing else is treated as 0 if you try to add a number to it. It's an empty string if you us…
> I'm a Python ignoramus, but I think they're the same, except for Perl's being more concise. Isn't everything always more concise in Perl? No matter what language it's compared to :) > How would you do a 3-level hash though in Python ? You might/should be able to wrangle that with some customisation of the default_factory or a defaultdict subclass, but yeah the arbitrary depth wouldn't be as 'plug n play' as with Pe…
> Isn't everything always more concise in Perl? No matter what language it's compared to :)
Ha, you need more exposure to things like APL :)