Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python idioms I wish I'd learned earlier

prooffreaderplus.blogspot.com

Post reply on HN
Print view View original

31–40 of 174 posts

Re: Python idioms I wish I'd learned earlier

#31

Post by dalke »

Animats wrote: post #13
euphemize wrote: post #9

One of my favorites: >>> print "* "* 50 to quickly print a separator on my terminal :) Previous discussion on python idioms from 300 days ago: https://news.ycombinator.com/item?id=7151433

That's cute, but the result of a bad design decision. Python overloads "+" as concatenate for strings. This also applies to lists. So [1,2,3] + [4,5,6] yields [1,2,3,4,5,6] This is cute, but not what you want for numerical work. Then, viewing multiplication as repeated addition, Python gives us [1,2,3]*4 yields [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3] This is rarely what was wanted. Then there's numpy, which has its own…

You went from "not what you want for numerical work" to "generalizing that concept comes back to bite you". I don't think you can make that step.

I do non-numeric scientific computing. (Meaning, I touch numpy about once a year.) My own code does things like

    [0] * N  # could be replaced with something like numpy.zeros()
    
    [QUERIES_FPS] * 501
    
    to_trans = [None]*256  # constructing a 256 byte translation table
        # (I could use zeros(), but used None to force an exception
        # if I missed a cell)
    
    self.assertEqual(self._get_records(simple.record*2),
                     [(simple.title, simple.record)]*2)
        # I parse a string containing two records and test I should
        # be able to get the (id, record) for each one
    
    ["--queries", SIMPLE_FPS, "-k", "3", "--threshold",
       "0.8"] + self.extra_args  # Construct a command-line from 2 lists
These idioms also exist in the standard library, like:

    webbrowser.py:  cmdline = [self.name] + [arg.replace("%s", url)
    sre_compile.py: table = [-1] + ([0]*len(prefix))
    ntpath.py: rel_list = [pardir] * (len(start_list)-i) + path_list[i:]
    traceback.py: list = ['Traceback (most recent call last):\n']
                  list = list + format_tb(tb, limit)
So while I think you are correct, in that "+" causes confusion across multiple domains with different meaning for "+", I think the moral is that operating overloading is intrinsically confusing and should be avoided for all but the clearest of use cases.

There is no best generalization to "+". For example, if you pick the vector math meaning, then regular Python would have that:

    ["A", "B"] + ["C", "D"] == ["AC", "BD"]
which has its own logic, but is likely not what most people who haven't done vector math expect.

Re: Python idioms I wish I'd learned earlier

#32

Post by ghshephard »

Wow - that's really, really great list.

In particular, #7 is something that I didn't even know existed, and I've been hacking around for 2+ years.

Instead of:

   mdict={'gordon':10,'tim':20}
   >>> print mdict.get('gordon',0)
   10
   >>> print mdict.get('tim',0)
   20
   >>> print mdict.get('george',0)
   0
I've always done the much more verbose:

   class defaultdict(dict):

       def __init__(self, default=None):
           dict.__init__(self)
           self.default = default

       def __getitem__(self, key):
           try:
               return dict.__getitem__(self, key)
           except KeyError:
               return self.default

   mdict=defaultdict(0)
   mdict['gordon']=10
   mdict['tim']=20
   print mdict['gordon']
   10
   print mdict['tim']
   20
   print mdict['george']
   0
I'll be sure to make great use of the dictionary get method - I'm embarrassed to admit how many thousands of times I could have used that, and didn't know it existed.

Re: Python idioms I wish I'd learned earlier

#33

Post by ubernostrum »

TheEzEzz wrote: post #26
frewsxcv wrote: post #17

Earlier quoted context omitted.

"This is rarely what was wanted." I don't know what else you would have expected...

[4,8,12]?

If you want to perform an operation on each item of an iterable, do that :)

[n * 4 for n in [1, 2 3]]

or

map(lambda n: n * 4, [1, 2, 3])

Re: Python idioms I wish I'd learned earlier

#34

Post by davidrusu »

hasenj wrote: post #18
kazinator wrote: post #14

Earlier quoted context omitted.

Common Lisp (and other dialects): ( Also: (lcm a b c d ...) ;; lowest common multiple (+) -> 0 (+ a) -> a (+ a b) -> a + b (+ a b c) -> (a + b) + c (*) -> 1 (* a) -> a (* a b) -> a * b (* a b c) -> (a * b) * c Is it just syntactic sugar? ( (and ( isn't the same as ( By the way, this could be turned into a short-circuiting operator: more semantic variation. Suppose < is allowed to control evaluation. Then an expressio…

But can any lisp dialect do: a = c ?

(= b c)) same number of operators but a few extra parens

Re: Python idioms I wish I'd learned earlier

#35

Post by ubernostrum »

Animats wrote: post #20
wodenokoto wrote: post #16

Earlier quoted context omitted.

Then what would you have [1,2,'q',[1,('a',2)]] + 4 yield? The reason why numpy lets you do math operation on each element in an array is because you can safely assume that each element is a number. You can assume absolutely nothing about the types of the elements in a list.

"TypeError: cannot add 'str' and 'int' objects." Just because you can define semantics for nonsense doesn't mean you should.

You apparently want lst * intval to be equivalent to map(lambda n: n * intval for n in lst) or [n * intval for n in lst]. Since Python has a convenient built-in and even syntactic sugar for doing what you want, why not let the operator overloading handle a different case?

(also, your issue is not with "nonsense semantics", it's with "my idea of how this operator should've been overloaded is different from their idea", and perhaps is even a beef with the idea of operator overloading in general, though if you like numpy I think you wouldn't like losing operator overloading)

Re: Python idioms I wish I'd learned earlier

#36

Post by philsnow »

Animats wrote: post #13
euphemize wrote: post #9

One of my favorites: >>> print "* "* 50 to quickly print a separator on my terminal :) Previous discussion on python idioms from 300 days ago: https://news.ycombinator.com/item?id=7151433

That's cute, but the result of a bad design decision. Python overloads "+" as concatenate for strings. This also applies to lists. So [1,2,3] + [4,5,6] yields [1,2,3,4,5,6] This is cute, but not what you want for numerical work. Then, viewing multiplication as repeated addition, Python gives us [1,2,3]*4 yields [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3] This is rarely what was wanted. Then there's numpy, which has its own…

Even worse IMHO is the semantics of strings being implicitly iterable. Often it ends up that you're intending to iterate over something

    for item in orders:
        do_something_with(item)
So if `foo` is usually `[Order(...), Order(...), ...]` but due to a bug elsewhere, sometimes `foo` is "some string". Then you get a mysterious exception somewhere down in `do_something_with` or one of its callees at run time, and all because the above snippet calls do_something_with('s'), do_something_with('o'), etc.

In my experience, this behavior is so seldom what is wanted that it should be removable (with a from __future__ style declaration) or just off by default.

Re: Python idioms I wish I'd learned earlier

#37

Post by bhy »

ghshephard wrote: post #32

Wow - that's really, really great list. In particular, #7 is something that I didn't even know existed, and I've been hacking around for 2+ years. Instead of: mdict={'gordon':10,'tim':20} >>> print mdict.get('gordon',0) 10 >>> print mdict.get('tim',0) 20 >>> print mdict.get('george',0) 0 I've always done the much more verbose: class defaultdict(dict): def __init__(self, default=None): dict.__init__(self) self.default…

Do you know there's also collections.defaultdict ?

Re: Python idioms I wish I'd learned earlier

#38

Post by dalke »

Animats wrote: post #20
wodenokoto wrote: post #16

Earlier quoted context omitted.

Then what would you have [1,2,'q',[1,('a',2)]] + 4 yield? The reason why numpy lets you do math operation on each element in an array is because you can safely assume that each element is a number. You can assume absolutely nothing about the types of the elements in a list.

"TypeError: cannot add 'str' and 'int' objects." Just because you can define semantics for nonsense doesn't mean you should.

I'll modify the example slightly to something which doesn't have a type error:

    [1,2,'q',[1,('a',2)]] * 4
With element-by-element operations, that would be

    [1 * 4,2 * 4,'q' * 4,[1,('a',2)] * 4]
giving

    [4, 8, 'qqqq', [1 * 4, ('a', 2) * 4]]
applying that again, assuming that tuple * scalar is also applied element-wise gives:

    [4, 8, 'qqqq', [4, ('a' * 4, 2 * 4)]]
and ends up with

    [4, 8, 'qqqq', [4, ('aaaa', 8)]]
I can't think of any case where that's meaningful.

Also, what should this do:

    x = [1]
    x.append(x)
    print(x + x)
    print(x * 4)
? Currently these print:

    [1, [1, [...]], 1, [1, [...]]]
    [1, [1, [...]], 1, [1, [...]], 1, [1, [...]], 1, [1, [...]]]
because the print function knows how to handle recursive definitions. Do all of the element-wise operations need to handle cyclical cases like this? I think numpy can get away with not worrying about this precisely because, as wodenokoto pointed out, it can assume a flat structure.

Re: Python idioms I wish I'd learned earlier

#39

Post by ghshephard »

bhy wrote: post #37
ghshephard wrote: post #32

Wow - that's really, really great list. In particular, #7 is something that I didn't even know existed, and I've been hacking around for 2+ years. Instead of: mdict={'gordon':10,'tim':20} >>> print mdict.get('gordon',0) 10 >>> print mdict.get('tim',0) 20 >>> print mdict.get('george',0) 0 I've always done the much more verbose: class defaultdict(dict): def __init__(self, default=None): dict.__init__(self) self.default…

Do you know there's also collections.defaultdict ?

I do now! I had known (and used) collections.OrderedDict, but had never used defaultdict. Millions of keyerrors later...

I clearly am going to have to spend a few hours today grokking everything that collections.* has to offer. Thanks very much.

Re: Python idioms I wish I'd learned earlier

#40

Post by philsnow »

smoe wrote: post #24
ckuehl wrote: post #10

> There is a solution: parentheses without commas. I don't know why this works, but I'm glad it does. It's worth mentioning that this is a somewhat controversial practice. Guido has even discussed removing C-style string literal concatenation: http://lwn.net/Articles/551438/ You may wish to consult your project's style guide and linter settings before using it.

I personally don't like this style of using multiple strings. Makes radical changes of the text cumbersome. I think in most cases it's better to use triple quotes. And if the content of these variables isn't exclusively shown in the shell, you should use translation files anyway.

    $ cat triple.py
    def foo():
        print """this is a triple quoted string
                 this is a continuation of a triple quoted string"""

    if __name__ == '__main__':
        foo()
    $ python triple.py
    this is a triple quoted string
                     this is a continuation of a triple quoted string
This is really warty. In bash you can mostly get around this with e.g.

    $ function usage() {
            cat
Post reply on HN