Live data from Hacker News

Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

codeconfessions.substack.com

21–30 of 101 posts

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#21

Earlier quoted context omitted.

I've never understood peoples confusion about function defaults. They are part of function definition, they aren't in the function body. You should expect them to be "executed" when the function is defined.

Or they could have been part of the function call, and run when the function is called without that argument.

You want Python to be how much slower now?

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#22
post #16

Using * on a sequence wouldn't get past code review with me, it's an obscure feature with surprising side-effects and it doesn't explain your thinking. Only context I've ever found it useful in, is if you're unit testing something that accepts a string of max length M, then bad_str = "A" * (M + 1) Is quite nice and also, easy to interpret. But if you're calling multiply on a list[list[Any]] lol nah mate, what the hel…

> Using * on a sequence wouldn't get past code review On a mutable sequence. Or anything mutable. [] being a sequence is not the issue here. If you work with immutable stuff, * is really not an issue.

It's not great though. I do use it to repeat strings but it should have been a method and not weirdly overloading *. This type of misuse of operator overloading is why people from a C++ background think operator overloading is evil. In python it's mostly ok, due to a more restrained culture, but this is a good example of when it's bad.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#23
post #14

Earlier quoted context omitted.

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

I really wonder what "more sane" languages you mean. Because a single [] does give you an empty list value in Python: one list, and not four different empty lists. Why would it give your four? That'd need to somehow bring in the surrounding context and in "sane" languages, (closed) expressions generally don't change the meaning because the context around them is changing. Actually, if you really want to have re-evalu…

F#:

    List.init 4 (fun _ -> [])

    [for _ in 1..4 -> []]
Or any sane language that uses immutable values by default like Elixir, Erlang, Clojure, Racket, Scheme, OCaml, etc.

> Actually, if you really want to have re-evaluting (sub)expressions, then Python does give you an option: comprehensions.

That's the complaint, that [[]]*4 is not shorthand for list comprehensions and that it's something entirely different. This is common in Python where two things you'd expect to be the same are indeed different in subtle ways.

List comprehensions in Python are closer to sane, but they still have unexpected behavior because of the way scoping works in Python.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#24

I think anybody that ever did LeetCode/HackerRank in Python had that bug at least once when making an empty 2-D DP table or a return matrix. My bigger question is how does a professor of machine learning does not know about one of the oldest Python footguns to exist: https://stackoverflow.com/questions/12791501/why-does-this-c... Very nice article otherwise, I never actually bothered looking into why it behaved this…

Yep, common enough that everyone learns it.

I started using loops and it motivated me to start writing list comprehensions. One of the foot guns that will really motivate you to understand Python as a tool more than anything.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#25
post #14

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The…

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

Languages with immutable data structures are relatively rare. With its high level focus python could have been one of them, but went for different trade offs.

> In more sane languages, the expression [] means give me an empty list value, nothing more and nothing less

What is an “empty list value”?

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#26
post #21

Earlier quoted context omitted.

Or they could have been part of the function call, and run when the function is called without that argument.

You want Python to be how much slower now?

This isn't about what I want, just noting that the other way around could sound just as intuitive. Anyway, it's awfully slow already, how much can it hurt ;-)

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#27
post #14

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The…

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

> I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that.

I think the small example is misguiding the intuition here.

If I write `my_list = f(expensive operation); long_list = my_list * 4`, I don't expect python to recreate the whole thing four times. I justly assume `f` will be evaluated once, and then copied.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#28
post #14

Earlier quoted context omitted.

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

> I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. I think the small example is misguiding the intuition here. If I write `my_list = f(expensive operation); long_list = my_list * 4`, I don't expect python to recreate the whole thing four times. I justly assume `f` will be evaluated once, and then copied.

That's a different statement and expected, but in a language like Python, it isn't always clear whether my_list is a value or a reference. And because of that, it leads to unexpected behavior when an intuitively expected value is actually a reference.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#29
post #14

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The…

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

Languages with immutable data structures are relatively rare. With its high level focus python could have been one of them, but went for different trade offs.

> In more sane languages, the expression [] means give me an empty list value, nothing more and nothing less

What is a “list value”? It’s either a pointer to some other space where the lists values are stored, or it’s the values stored themselves. If you say it’s the second (I can’t think of any language that does), then `a = […]; b = a` is an expensive copy by value assignment. Is that worth that trade off to get less surprising results in the articles case? I’m not saying the answer is “obviously no”, but it definitely isn’t “obviously yes”.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#30
post #14

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The…

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

Agreed. Python is aimed at developers, not language enthusiasts, and you should be able to intuit what something is doing without having to memorize dozens of obscure rules and edge cases. The less surprises a language has, the better the design.
Post reply on HN