Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python built-ins worth learning

treyhunner.com

Post reply on HN
Print view View original

61–69 of 69 posts

Re: Python built-ins worth learning

#61

Post by dTal »

grepgeek wrote: post #58
dTal wrote: post #56

Earlier quoted context omitted.

I also spent years writing Python scripts without a single class. I didn't really understand the point of objects at all. I used them because they were part of library interfaces, but I never created them. The scripts didn't have to manage much state, so writing everything in a pure-functional style worked fine. Nobody could really explain to me what objects were all about - all the arguments seemed to apply equally…

You could have still kept the massive state in a dict or a list or a tuple and passed that dict around from one function to another, could you not? Why did it become necessary to implement classes?

Because I don't want to "pass around" the state - I want to hide it. Yes, of course it's possible to mingle the state in all the rest of the program, just like it's possible to scatter gotos everywhere instead of using structured control flow. But what I really want is to call EnablePowerSupply(), rapidly followed by SetVoltage(30), and have all the messy business of statefully talking over a serial port (and not having commands stomp on each other) neatly abstracted away. EnablePowerSupply and SetVoltage need to share state to do that. That could indeed be done by passing an extra parameter - EnablePowerSupply(blobOfState) and SetVoltage(blobOfState, 30) - but that's basically exactly what objects are syntactic sugar for in Python. Only blobOfState is more usually called "self".

Oh, and of course there's not one, but half a dozen power supplies. You could pass around the blobs of state seperately of course, but now you have to manage their scope independently from the functions that operate on them - a useless decoupling that adds overhead. What I ended up with was something like:

[psu.enable() for psu in psus]

Which you can always do, whenever psus is within scope. Hard to get terser and more idiomatic than that.

Re: Python built-ins worth learning

#62

Post by nathanasmith »

tehnub wrote: post #57
abhishekjha wrote: post #48

Earlier quoted context omitted.

Forgive me doesn’t deque mean double ended queue? Also why can’t a simple array behave like a list?

Yes it does mean double ended queue. I'll assume you mean a python list when you say simple array. One drawback of lists is that you cannot add a new element to the start of a list in constant time (with a python list it requires O(n) time), whereas you can with a deque.

I had a script a while back that created really long lists at runtime by continually appending data as it came over the wire. The lists would quickly get so long that I needed to remove items from the beginning to conserve memory though because reasons I could only remove items one at a time. Long story short, converting those lists to collections.deque and making use of popleft() rather than "del l[0]" improved performance considerably.

Re: Python built-ins worth learning

#63

Post by ForHackernews »

joatmon-snoo wrote: post #40
Waterluvian wrote: post #30

Sets are my #1 favourite Python built-in. I used to have these tedious imperative functions for doing change detection on collections. Then I learned sets and it turned into obvious code like: added = b - a removed = a - b repeated = a & b Etc. Of course this is set theory and not Python specific. But still. Learn sets!

frozenset is a really nice one for enforcing immutabolity :)

Frozensets can also be keys in a dictionary.

Re: Python built-ins worth learning

#64

Post by psalminen »

MattConfluence wrote: post #55

A recent(-ish) addition to Python 3 I also think is worth mentioning is the statistics module: https://docs.python.org/3/library/statistics.html Every codebase eventually needs an averaging function, now you don't have to re-implement it yourself every time. Plus it's a nice toolbox of a few different utilities that you would otherwise need to install numpy/scipy/etc. to get.

Didn't know about this, thanks!

Re: Python built-ins worth learning

#65

Post by pfranz »

grepgeek wrote: post #58
dTal wrote: post #56

Earlier quoted context omitted.

I also spent years writing Python scripts without a single class. I didn't really understand the point of objects at all. I used them because they were part of library interfaces, but I never created them. The scripts didn't have to manage much state, so writing everything in a pure-functional style worked fine. Nobody could really explain to me what objects were all about - all the arguments seemed to apply equally…

You could have still kept the massive state in a dict or a list or a tuple and passed that dict around from one function to another, could you not? Why did it become necessary to implement classes?

That is one way of writing programs. Python is very OO friendly. If any of the main features of OO make your code more maintainable; inheritance, polymorphism, encapsulation, overloading, then use those. Passing around a dict is an object, but it's often nice to keep methods for interacting with that object along with the data (for reasons mentioned above).

Re: Python built-ins worth learning

#66

Post by ehsankia »

abhishekjha wrote: post #48
ehsankia wrote: post #45

Earlier quoted context omitted.

I use defaultdict so damn often, it honestly should even be a builtin in my opinion. You also forgot `itertools`, which imo is even more useful than `functools`. I use `chain` and `groupby` quite often. There's also `collections.deque` which is a quick linked list implementation if you need either a stack or a queue.

Forgive me doesn’t deque mean double ended queue? Also why can’t a simple array behave like a list?

Normal lists have a backing array. In theory, every time you resize it, it needs to allocate new memory and copy everything over, which is very slow. Obviously python does a lot of optimization behind the scene by over-allocating memory, which gives it better amortized speed.

Linked lists are slow at accessing data in the middle, but you can very quickly add and remove stuff from the ends, hence double-ended queue. A stack is basically a deque where you only use the tip, so deque is very versatile like that.

Re: Python built-ins worth learning

#67

Post by mos_basik »

dTal wrote: post #61
grepgeek wrote: post #58

Earlier quoted context omitted.

You could have still kept the massive state in a dict or a list or a tuple and passed that dict around from one function to another, could you not? Why did it become necessary to implement classes?

Because I don't want to "pass around" the state - I want to hide it. Yes, of course it's possible to mingle the state in all the rest of the program, just like it's possible to scatter gotos everywhere instead of using structured control flow. But what I really want is to call EnablePowerSupply(), rapidly followed by SetVoltage(30), and have all the messy business of statefully talking over a serial port (and not hav…

I know it's a contrived example, but it's worth pointing out that using comprehensions to cause side effects is considered by some to be at least unPythonic and at worst an abuse of the construct.

I know this because I wanted to do the same thing and have looked all over for a justification for it. In this case most people seem to agree that the bog standard for loop is the way to go:

    for psu in psus:
        psu.enable()
Some people (my boss) will even put it on a single line, so you don't lose much terseness.

I think the rule of thumb is don't use a list comprehension unless you're going to use the list afterwards, else you're wasting an allocation.

Re: Python built-ins worth learning

#68

Post by lsorber »

MattConfluence wrote: post #55

A recent(-ish) addition to Python 3 I also think is worth mentioning is the statistics module: https://docs.python.org/3/library/statistics.html Every codebase eventually needs an averaging function, now you don't have to re-implement it yourself every time. Plus it's a nice toolbox of a few different utilities that you would otherwise need to install numpy/scipy/etc. to get.

Thanks, hadn't seen this before!

I wonder why they didn't have just one median function and control its behaviour with keyword arguments instead of creating 4 different median functions.

Re: Python built-ins worth learning

#69

Post by dTal »

mos_basik wrote: post #67
dTal wrote: post #61

Earlier quoted context omitted.

Because I don't want to "pass around" the state - I want to hide it. Yes, of course it's possible to mingle the state in all the rest of the program, just like it's possible to scatter gotos everywhere instead of using structured control flow. But what I really want is to call EnablePowerSupply(), rapidly followed by SetVoltage(30), and have all the messy business of statefully talking over a serial port (and not hav…

I know it's a contrived example, but it's worth pointing out that using comprehensions to cause side effects is considered by some to be at least unPythonic and at worst an abuse of the construct. I know this because I wanted to do the same thing and have looked all over for a justification for it. In this case most people seem to agree that the bog standard for loop is the way to go: for psu in psus: psu.enable() So…

Thanks for the heads up. I'm not totally convinced about "Pythonic" as a figure of merit for anything (often it seems to merely mean "clunky"), but it's a good point about the wasted allocation [1]. And the one-liner for-loop is almost identical to the comprehension anyway.

I think the reason I instinctively write it this way is because in my mind, I basically think of list comprehensions as sugar for map + lambda. I'm "really" trying to write map(enable, psus). But of course, it's a method, so you need the instance[2] - map(lambda psu: psu.enable(), psus). The reason I prefer a map over a for loop is because it's a habit borne of the principle of least power - map provides a guarantee than no "funny business" (data dependency) is going on between the elements of the list you're iterating over. I scrupulously avoid for loops on principle, unless I need that kind of funny business. Of course in this case the for loop is so short as to make no difference, but like I say - it's a habit. In my code, "for" means "funny business here".

[1] not that it matters in this case - you're not toggling power supplies in a tight loop.

[2] Technically, in Python, map(psus[0].enable, psus) would work if psus was not empty. Or you could spawn a new instance: map(PSU().enable, psus). But ugh, talk about defeating the purpose.

Post reply on HN