Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Python through its builtins

sadh.life

Post reply on HN
Print view View original

51–60 of 180 posts

Re: Understanding Python through its builtins

#51

Post by nneonneo »

sireat wrote: post #43

I am a pretty average Python programmer(5 years teaching, 15 years writing). I still wonder what was the reasoning for allowing creation of local objects with the same name as builtins. Okay it can be nice to redefine pprint as print I suppose. Still how many sum, list, min, max, dict(!) have been erroneously redefined in beginner tutorials and beginner code. From my experience sum and list suffer the most. Sure ther…

Python itself doesn’t disallow this because there are quite a lot of builtins with useful names - for example, `file`, `id`, and `hash` to name a few. Disallowing setting these would be tantamount to adding a bunch of new keywords to the language, which they’ve been quite loathe to do in general.

A good linter will catch these, so in production environments you usually don’t run into issues. I agree that it can be a beginner trap though!

Re: Understanding Python through its builtins

#52

Post by adrusi »

Blikkentrekker wrote: post #19
radarsat1 wrote: post #5

Earlier quoted context omitted.

> (things like `.append` changing the object, returning `None` instead of creating a copy and returning that) This would be horrendously inefficient without immutable data structures like Clojure's. Very few languages have that, so it's a strange assumption to make, especially for a language as old as Python. Although it is a very nice feature of Clojure.

It should also be worth nothing that Clojure sacrificed quite a bit to make this as efficient as possible. “persistent vectors” are certainly an interesting data structure that strike a compromise between fast indexing and being able to relatively quickly create a copy where only one element changes, but it's a compromise and indexing is made slower to allow for the latter. — They also take up more memory on their ow…

Random access into a clojure vector is going to need more memory lookups than conventional sequential buffer array (I don't recall the constants used in the implementation, I think it's either 4 or 8 lookups).

But when you're indexing into the vector sequentially, the memory layout plays rather well with memory caching behavior, and most lookups are going to be in L1 cache, just like they would be in a conventional array.

So lookups are a bit more expensive, but not as much more expensive as one might imagine.

Re: Understanding Python through its builtins

#53

Post by wpietri »

sireat wrote: post #43

I am a pretty average Python programmer(5 years teaching, 15 years writing). I still wonder what was the reasoning for allowing creation of local objects with the same name as builtins. Okay it can be nice to redefine pprint as print I suppose. Still how many sum, list, min, max, dict(!) have been erroneously redefined in beginner tutorials and beginner code. From my experience sum and list suffer the most. Sure ther…

No idea what their actual reasoning is, but here's how I think about it:

This is better for novices, because otherwise you create a whole bunch of land mines for people who are desperately trying to get something done. If they aren't aware of the built-in then they aren't trying to use it. Insisting that they become aware of something they don't want right then will be frustrating.

It's also better for experts, in that they're generally aware they're overriding a built-in and are doing it on purpose, and if not they'll have an IDE or linter reminding them.

To me, I see tooling as a spectrum from supportive to controlling. Python is very much on the supportive end. It feels controlling when I get interrupted because some programmer who has never met me programmed a tool to insist I do things their way. That would very much include insisting I respect a bunch of names they decided long ago to put in the global namespace.

Re: Understanding Python through its builtins

#54

Post by Blikkentrekker »

adrusi wrote: post #52
Blikkentrekker wrote: post #19

Earlier quoted context omitted.

It should also be worth nothing that Clojure sacrificed quite a bit to make this as efficient as possible. “persistent vectors” are certainly an interesting data structure that strike a compromise between fast indexing and being able to relatively quickly create a copy where only one element changes, but it's a compromise and indexing is made slower to allow for the latter. — They also take up more memory on their ow…

Random access into a clojure vector is going to need more memory lookups than conventional sequential buffer array (I don't recall the constants used in the implementation, I think it's either 4 or 8 lookups). But when you're indexing into the vector sequentially, the memory layout plays rather well with memory caching behavior, and most lookups are going to be in L1 cache, just like they would be in a conventional a…

How? I don't see how that's possible.

The actual data of a Pvector is not in contiguous memory but scattered however the JVM wills it, and on top of that in order to find which address to retrieve it, an algorithm that runs in logarithmic time with respect to the length of the vector must be used opposed to a constant time one.

How can most lookups end up in L1 cache if an element that is 32 indices removed is statistically likely to be arbitrarily far removed in memory?

Of course, all of that is not that material to begin with given that most elements will be pointers to begin with so the actual objects wither they point will already be arbitrarily scattered and it simply adds one more pointer indirection, but for unboxed types such as integers it does play a factor.

Re: Understanding Python through its builtins

#55

Post by hultner »

DangitBobby wrote: post #50

Very well written. Fun little tidbit, Django abuses the fact that bools are ints in it's partition util: https://github.com/django/django/blob/01bf679e59850bb7b3e639...

Where? I don't see anything in that code relying on that unless I'm Sunday blind.

Re: Understanding Python through its builtins

#56

Post by hultner »

eximius wrote: post #28

In addition to this, I highly recommend just reading the codebase. I haven't written C since college and it's remarkably readable. I once tried to catalogue all the stdlib operations which release the GIL, meaning if you use only those (well, only those "heavy" bits, you can still use other small blocking glue bits), you can do "real" multithreading. It was a fun exercise!

There's a really nice (although old now) walk through of the cpython code base on YouTube. I watched it on a long 24 hour flight between Canada and Sweden a couple of years back.

Edit: Found it! You're in for about 9 hours of quality watching. https://youtube.com/playlist?list=PLwyG5wA5gIzgTFj5KgJJ15lxq...

Re: Understanding Python through its builtins

#57

Post by iezepov »

hultner wrote: post #55
DangitBobby wrote: post #50

Very well written. Fun little tidbit, Django abuses the fact that bools are ints in it's partition util: https://github.com/django/django/blob/01bf679e59850bb7b3e639...

Where? I don't see anything in that code relying on that unless I'm Sunday blind.

`results[predicate(item)]` here they get the first and the second elements of a tuple. Essentially it’s `results[False]` and `results[True]`

Re: Understanding Python through its builtins

#58

Post by hultner »

iezepov wrote: post #57
hultner wrote: post #55

Earlier quoted context omitted.

Where? I don't see anything in that code relying on that unless I'm Sunday blind.

`results[predicate(item)]` here they get the first and the second elements of a tuple. Essentially it’s `results[False]` and `results[True]`

Oh, that's obvious now. Must be that Sunday night blindness.

Re: Understanding Python through its builtins

#59

Post by NegativeLatency »

hultner wrote: post #55
DangitBobby wrote: post #50

Very well written. Fun little tidbit, Django abuses the fact that bools are ints in it's partition util: https://github.com/django/django/blob/01bf679e59850bb7b3e639...

Where? I don't see anything in that code relying on that unless I'm Sunday blind.

Results has 2 elements indexed by 0 and 1 (it’s a tuple not a dict)

Re: Understanding Python through its builtins

#60

Post by tusharsadhwani »

nneonneo wrote: post #49

Cute fact about __debug__: it is one of the only ways to get compile-time conditionals in Python. Performing a comparison with `if __debug__:` will output byte code for the ensuing statement if and only if the interpreter is in debug mode - notably, in `-O` mode, it will not even generate a load of __debug__ and a conditional jump, and acts as if the statement didn’t exist at all.

that is indeed interesting. Mind if I add this in the article?
Post reply on HN