Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

codeconfessions.substack.com

Post reply on HN
Print view View original

81–90 of 101 posts

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#81

Post by hughesjj »

bmitc wrote: post #41
magnio wrote: post #37

Earlier quoted context omitted.

> In more sane languages, the expression [] means give me an empty list value, nothing more and nothing less. So then whatever the syntax for [[]]*4 is in such sane languages, it gives you four empty list values. I don't buy this. Either the syntax to give you four empty list values is some kind of comprehension, which Python has and does exactly what you like, or it must have lazy evaluation somehow. Otherwise, how…

My question is: what's a real-world example where this [...]*n syntax buys you something that can't be done simply otherwise and that is worth the complexity tradeoff of people often using it subtly wrong?

It's useful for math when doing scalar multiplication but imo it would make more sense to use a tuple instead of list

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#82

Post by hughesjj »

bmitc wrote: post #14
kerkeslager wrote: post #6

I like the deep dive into the source, but big picture, I think if you understand the general order of evaluation which applies to most languages, this isn't all that unintuitive. The arguments going into the outer [ ] get evaluated before the outer [ ], meaning the inner [] gets evaluated first. Once that happens the inner list is created, and there's no code left that even visually could create more inner lists. The…

You can whitewash this away by discussing order of evaluation, but it still is a great example of how Python continually mixes up concepts of expressions and statements and immutable and mutable data. In real life, if I ask someone to go get me four empty buckets, I expect them to come back with four distinct empty buckets. The intuitive interpretation of [[]]*4 is asking just that. Instead Python creates a bucket an…

> immutable and mutable data

I don't get how this is an example of python mixing up mutable/immutable data, and I'm unaware of a situation where python does. Conceptually I'm not even sure what you mean by a language 'mixing up mutable and immutable data' unless it's literally just a buggy/non hardened implementation.

A = (1, ); A[0] = 2 # this raises an exception

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#83

Post by dragonwriter »

zarzavat wrote: post #63
bmitc wrote: post #28

Earlier quoted context omitted.

That's a different statement and expected, but in a language like Python, it isn't always clear whether my_list is a value or a reference. And because of that, it leads to unexpected behavior when an intuitively expected value is actually a reference.

Lists in Python are always references. There’s a separate type called tuple specifically designed to represent lists-as-values. If you do [()]*4 then you get exactly what you want.

> There’s a separate type called tuple specifically designed to represent lists-as-values.

tuples are also references, but to an immutable object.

> If you do [()]*4 then you get exactly what you want.

I think what is actually wanted here is four different mutable lists rather than the same mutable list four times, something like what you would get from:

  [[] for _ in range(4)]

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#84

Post by klyrs »

kerkeslager wrote: post #71
klyrs wrote: post #21

Earlier quoted context omitted.

You want Python to be how much slower now?

This is a terrible excuse. Quickly doing the wrong thing isn't optimization.

The language spec isn't wrong, you just don't like it.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#85

Post by bmitc »

mapreduce wrote: post #54
bmitc wrote: post #41

Earlier quoted context omitted.

My question is: what's a real-world example where this [...]*n syntax buys you something that can't be done simply otherwise and that is worth the complexity tradeoff of people often using it subtly wrong?

> that is worth the complexity tradeoff of people often using it subtly wrong? I don't think that makes the complexity tradeoff worth it. This is the kind of tradeoff that makes the language implementation, reference and semantics more and more complex. Unfortunately Python has a lot of such complexities already. We don't need more of it. Such complexity hurts the creation of completing implementations. IMHO programm…

> these people need to roll up their sleeves and just learn the damn language

You really have no choice but to do that. But the critique here is that some languages make this hard. And some languages, like Python, appear deceptively simple and consistent, when they are anything but. And as I pointed out, these decisions were not really required or designed to solve certain problems. They just kind of came about in Python's development, one historically and intentionally ignorant of pre-existing languages and their good ideas.

When everybody just says "learn the language" or "there's list comprehensions" or "there's for loops", then why does the [...]*n syntax exist? What problems is it solving that require the confusion that it generates?

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#86

Post by bmitc »

dragonwriter wrote: post #77
bmitc wrote: post #28

Earlier quoted context omitted.

That's a different statement and expected, but in a language like Python, it isn't always clear whether my_list is a value or a reference. And because of that, it leads to unexpected behavior when an intuitively expected value is actually a reference.

> That's a different statement and expected, but in a language like Python, it isn't always clear whether my_list is a value or a reference It is always clear, and it is 100% always a reference, with a set of cases where that reference is to a singleton or immutable object, and a subset of those cases where, in particular Python implementations, that singleton may effectively be implemented as a special case value. I…

And again, that decision is the more complex one. Inverting that such that everything is immutable by default and mutable or a reference by choice is the more regular, consistent choice that removes a fairly big swath of common programming problems.

The critique here is that it's a badly designed language in that it's design decisions often get in the way of writing software that works as you expect. We're not discussing about whether you can learn such a system. The discussion is a lament on why it had to be this way.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#87

Post by dragonwriter »

bmitc wrote: post #86
dragonwriter wrote: post #77

Earlier quoted context omitted.

> That's a different statement and expected, but in a language like Python, it isn't always clear whether my_list is a value or a reference It is always clear, and it is 100% always a reference, with a set of cases where that reference is to a singleton or immutable object, and a subset of those cases where, in particular Python implementations, that singleton may effectively be implemented as a special case value. I…

And again, that decision is the more complex one. Inverting that such that everything is immutable by default and mutable or a reference by choice is the more regular, consistent choice that removes a fairly big swath of common programming problems. The critique here is that it's a badly designed language in that it's design decisions often get in the way of writing software that works as you expect. We're not discus…

> And again, that decision is the more complex one. Inverting that such that everything is immutable by default and mutable or a reference by choice is the more regular, consistent choice that removes a fairly big swath of common programming problems.

Maybe. But that's largely irrelevant since we are discussing the impact of a syntax choice regarding an operator on an explicitly-mutable container type, and specifically the behavior of it used when the container also contains a member of the same type.

And the alleged surprising thing is not the mutability of anything, but thar after the operation, all the members of the other container are the same instance of the inner mutable container type.

Python being designed ground up with immutability by default and explicitly opt-in mutability would not directly have any bearing on this issue, as long as it still had the capacity to express immutable lists and the capacity to apply something like the list-reptition operation to them. Except it would probably make all of the involved examples more verbose.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#88

Post by kerkeslager »

aeonik wrote: post #72
kerkeslager wrote: post #69

Earlier quoted context omitted.

This feels like a legitimate critique to me. I'm developing a programming language myself and I'm not going to include a syntax like this--multiplying an array by a number feels like a type error to me.

I wouldn't recommend just throwing the idea away, it just needs to be approached with care. You can do it in a type safe way. Research Rank Polymorphism and Array Languages: https://en.m.wikipedia.org/wiki/Array_programming https://arxiv.org/abs/1907.00509

I mean, is there anything there that couldn't just be a library function? Not everything has to be an operator.

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#89

Post by kerkeslager »

klyrs wrote: post #84
kerkeslager wrote: post #71

Earlier quoted context omitted.

This is a terrible excuse. Quickly doing the wrong thing isn't optimization.

The language spec isn't wrong, you just don't like it.

The language spec isn't right you just like it.

Sure, there are a lot of subjective aesthetics that go into the spec, but in this case, there are objective reasons for not liking this. It's a well-known footgun that causes bugs. And it's almost never what you want, so you end up doing something like this:

    def f(xs = None):
        # Are these two lines actually faster than the
        # interpreter creating defaults at call time?
        if xs is None:
            xs = []

        ...
Do you have any reasons at all for defending this decision?

Re: Why Do Python Lists Multiply Oddly? Exploring the CPython Source Code

#90

Post by mapreduce »

bmitc wrote: post #85
mapreduce wrote: post #54

Earlier quoted context omitted.

> that is worth the complexity tradeoff of people often using it subtly wrong? I don't think that makes the complexity tradeoff worth it. This is the kind of tradeoff that makes the language implementation, reference and semantics more and more complex. Unfortunately Python has a lot of such complexities already. We don't need more of it. Such complexity hurts the creation of completing implementations. IMHO programm…

> these people need to roll up their sleeves and just learn the damn language You really have no choice but to do that. But the critique here is that some languages make this hard. And some languages, like Python, appear deceptively simple and consistent, when they are anything but. And as I pointed out, these decisions were not really required or designed to solve certain problems. They just kind of came about in Py…

> What problems is it solving that require the confusion that it generates?

I think I answered that already. It keeps the language spec consistent and simpler.

Imagine the complexity you have to add to the language spec to say that when we write [] we deal with the reference to this list except in the multiplication syntax a = [[]] * 5 where the inner [] is not a reference to the list but the list value! Such special case will make the language both inconsistent and harder to understand for experienced programmers.

Post reply on HN