Live data from Hacker News

Writing and linting Python at scale

engineering.fb.com

121–130 of 160 posts

Re: Writing and linting Python at scale

#121
post #72
post #68

Earlier quoted context omitted.

> A variable int tuple, for example, can be typed as Tuple[int, ...]. That's a type that matches tuples of any length, not a variable-length tuple. The size of a tuple can't be changed. A variable-length tuple doesn't even really make sense, what you'd want there is a list. > You can't concatenate two variable-length tuples, which makes sense - where would the cutoff be? But you should absolutely be able to concatena…

> That's a type that matches tuples of any length, not a variable-length tuple. A tuple with a type that matches variable lengths of tuples is a variable-length tuple for that piece of code. You're free to show me some official definitions that proves this wording false, but until then it's useless nitpicking. Though you should probably take that up with Guido, who also calls them variable-length tuples: https://gith…

I think they've interpreted "variable length tuple" as "a tuple whose length can change", not "a tuple whose length could be one of multiple options".

The former is of course not possible with tuples being immutable, which is why they're talking about lists.

Re: Writing and linting Python at scale

#122
post #72

Earlier quoted context omitted.

> That's a type that matches tuples of any length, not a variable-length tuple. A tuple with a type that matches variable lengths of tuples is a variable-length tuple for that piece of code. You're free to show me some official definitions that proves this wording false, but until then it's useless nitpicking. Though you should probably take that up with Guido, who also calls them variable-length tuples: https://gith…

I think they've interpreted "variable length tuple" as "a tuple whose length can change", not "a tuple whose length could be one of multiple options". The former is of course not possible with tuples being immutable, which is why they're talking about lists.

Yeah, that seems likely. I'm not sure how to express the concept aside from the name I've seen used in the community (and my many subsequent explanations), considering I was explicitly talking about typing.

Re: Writing and linting Python at scale

#123
post #90
post #80

Earlier quoted context omitted.

> What's missing is in which context the tuple's length is variable, and in which context it is fixed. Simple example: a function has a parameter whose type is "variable-length tuple of int". You can pass any tuple in that is known to have 0..n elements, all of type int. What would you have me call that, other than the name I've seen used in discussions on this feature? > "Arbitrary" doesn't really help because it co…

I'd call that function: polymorphic over tuple length.

That kind of works. I guess I keep thinking about it from the perspective of the type itself instead of the function that uses it.

Re: Writing and linting Python at scale

#124
post #122

Earlier quoted context omitted.

I think they've interpreted "variable length tuple" as "a tuple whose length can change", not "a tuple whose length could be one of multiple options". The former is of course not possible with tuples being immutable, which is why they're talking about lists.

Yeah, that seems likely. I'm not sure how to express the concept aside from the name I've seen used in the community (and my many subsequent explanations), considering I was explicitly talking about typing.

I don't know either. Since the alternative interpretation is a contradiction in terms, you'd think this name would cover the intended meaning.

Maybe something like "unknown length tuple"? chatgpt suggested "arbitrary-length" or "undetermined-length" as synonyms, that could be a more easily understood expression perhaps.

Re: Writing and linting Python at scale

#125
post #122

Earlier quoted context omitted.

Yeah, that seems likely. I'm not sure how to express the concept aside from the name I've seen used in the community (and my many subsequent explanations), considering I was explicitly talking about typing.

I don't know either. Since the alternative interpretation is a contradiction in terms, you'd think this name would cover the intended meaning. Maybe something like "unknown length tuple"? chatgpt suggested "arbitrary-length" or "undetermined-length" as synonyms, that could be a more easily understood expression perhaps.

Those are good suggestions, thanks!

Re: Writing and linting Python at scale

#126

Earlier quoted context omitted.

No this is actually wrong. It works but it doesn't capture the meaning of the nature of a tuple. A Tuple is Typed as something as a Fixed size. That's right, it's like this at the Type level. The entire concept of a tuple is a Product type or essentially like a struct but with no names for each parameter. Tuple[int, str, float] #correct Doing what you're doing here is equivalent to creating a Struct with variadic pro…

> It works but it doesn't capture the meaning of the nature of a tuple. That's backwards. It doesn't work (as noted in GP, because Python supports unpacking only a single variadic parameter in a type annotation), but it does capture the nature of a tuple. It is (or, more precisely, would be , if the syntactic limitation was relaxed to allow it) a generic function that operates on two tuples of arbitrary tuple types a…

> a generic function that operates on two tuples of arbitrary tuple types and returns a tuple of a third tuple type whose shape is a simple concatenation of the shapes of the input tuples.

This isn't definable without dependent types or a concat operator at the type level. Dependent types aren't a thing we are talking about here and languages that support this feature are waaay out of scope.

You're basically asking for stuff like this:

   func concat(a: Array[int; M], b Array[int; N]) -> Array[int; N + M]
or

   func concat(a: Tuple[*args1], b Tuple[*args2]) -> Array[*(args + args)]
Which is bringing those programming terms up a level into types. This doesn't exist in the realm of programming we're talking about. Idris, Lean and Coq are more you're game here and those languages are clearly off topic.

>But... I don't want that, and that's not what this works on.

You want something well out of scope of this conversation and well out of scope of most practical programming languages. You still don't get what a tuple is. Nobody defines a Product type with an arbitrary amount of properties anymore then someone defines a struct with an arbitrary amount of properties anymore then they would do it for a tuple.

This is NOT the definition of a tuple. People are confusing the Tuple for some Immutable list.

>Python is very close to allowing that, which is very different than:

Python shouldn't allow this, neither should TS. If typescript allows it, it's probably some arbitrary shortcut feature. Dependent types at the type level supported in a very generic way is very very unlikely.

Like you people are trying to explain to me what a generic is while doing all kinds of programming operations on type level entities like concatenating them and combining shapes. Is the concat the only way to do it? What about an intersection? These operators only come into play with advanced theoretical langauges like Lean or Idris and it's very off topic.

Re: Writing and linting Python at scale

#127
post #118

Earlier quoted context omitted.

You're thinking fixed size arrays. It doesn't make sense to have tuples of arbitrary size. To make it arbitrary size you have to constrain it on a single generic parameter which makes it functionally equivalent to a fixed size array. You essentially just do this with an array or a list. Read my other reply to you to see the problem of taking functions with "generic" number of parameters. What are you going to do with…

> You're thinking fixed size arrays. It doesn't make sense to have tuples of arbitrary size. My god. No, Python doesn't support "fixed size arrays", but it does support tuples of arbitrary size, no matter how often you claim that to be wrong. > Read my other reply to you to see the problem of taking functions with "generic" number of parameters. What are you going to do with each parameter? What are the types of each…

>My god. No, Python doesn't support "fixed size arrays", but it does support tuples of arbitrary size, no matter how often you claim that to be wrong.

My god I never claimed this. I'm just talking about programming in general. I'm trying to show you that what YOU are asking for is programming operations to happen at the type level. You're not seeing it correctly.

>I could, for example, do the thing I've been writing dozens of replies asking for: concatenate two arbitrary tuples. It's a perfectly well-defined operation, and without an artificial limitation on unpacking of TypeVarTuples, it would already be possible.

You're not getting it. Ok let me spin it to you another way.

Imagine you Do have two tuples that you concat:

    function concat(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 + arg2)]:
Right? You notice the plus operator at the end there where I concatenate all the generic arguments and unroll it right? You're saying TS supports this and that it's common languages to support that "common" operation.

Well here's another common operation. What if I want the INTERSECTION of the two tuples?

   function intersect(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 | arg2)]:
You see what's going on here? You're asking for programming operators like concat and intersect and union INSIDE of types. You are asking for features that DON'T EXIST in practical type based languages. These features exist in academic languages. And those academic languages are completely off topic.

That's what I'm getting at. Read it and understand it.

>I could, for example, do the thing I've been writing dozens of replies asking for: concatenate two arbitrary tuples. It's a perfectly well-defined operation, and without an artificial limitation on unpacking of TypeVarTuples, it would already be possible.

You can do this and define a function that does this in python and likely typescript. But you would be EXITING the type system when you do this. That means using keywords like Any and potentially running into runtime errors.

I want to bold that part above. I'm pretty sure whatever you're doing either uses a bunch of Any's which DOES not preserve the shape fully. This is likely what's going on. It may pass the type check but it's not doing what you mentioned in your initial post which is Preserving the shape of the type.

I haven't used typescript. But IF what you say about typescript is TRUE and that it type checks the concatenation of shapes then it's likely an arbitrary one off feature. Have a look at those functions again:

       function concat(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 + arg2)]:
       function intersect(x: Tuple[*arg1], y: Tuple[*arg2]) -> Tuple[*(arg1 | arg2)]:
Basically you're saying that you want some language that supports type level programming where you can concat types, intersect types, divide types, likely bring programming terms up into the type level and have propositional terms like

      func add(x: int  int 
All of this is unlikely to be supported by any practical language. Likely TS has a one off where if you do this:

      func concat(x: Tuple[*args], y: Tuple[*args]) -> Tuple[M]:
           return x.concat(y) 
Where x.concat(y) is just special in that it is the only method that instantiates a typed tuple that is the concatenation of both tuples. Again I repeat it is unlikely for typescript to support what you are asking for in a very generic way where you can program your types.

Re: Writing and linting Python at scale

#128

Earlier quoted context omitted.

You're thinking fixed size arrays. It doesn't make sense to have tuples of arbitrary size. To make it arbitrary size you have to constrain it on a single generic parameter which makes it functionally equivalent to a fixed size array. You essentially just do this with an array or a list. Read my other reply to you to see the problem of taking functions with "generic" number of parameters. What are you going to do with…

> To make it arbitrary size you have to constrain it on a single generic parameter Your information is out of date since Python 3.11: https://peps.python.org/pep-0646

Bro, Look at this:

https://peps.python.org/pep-0646/#multiple-type-variable-tup...

Does this not exactly mean: "To make it arbitrary size you have to constrain it on a single generic parameter"

Re: Writing and linting Python at scale

#129
post #117

Earlier quoted context omitted.

>I'm not sure what you're trying to say here. Yes, you can implement the function, I never claimed otherwise. I'm talking about the _type system_. The type constrains the definition. Look at the definition, there is ONLY one possible definition when converting this: Dict[str, T] -> Dict[str, Callable[[], T]] Literally. Try to think of another way to define the lambda, you can't. The more open the type the more constr…

> The type constrains the definition. Look at the definition, there is ONLY one possible definition when converting this: Yes, and if Python had a better type system, I could write a more concrete definition. Why don't you understand that? > I remarked on this in other replies. How will you define the return value of the concat and have it exactly type correct? It's not possible bro. You can't do much with tuples of…

>Yes, and if Python had a better type system, I could write a more concrete definition. Why don't you understand that?

I understand everything. It's you who doesn't understand. What I described is for ALL type systems in general. Not python types.

>Of course it's possible, the type system even supports almost everything that's necessary. You can already define what I want with a single fixed-size tuple of arbitrary size, you just can't do it with multiple ones. Why do you have this strange fixation on calling this "wrong" and "impossible"?

It's not a fixation. It's because I know what I'm talking about.

Look just prove it to me. Define the type signature for concat function that will concat two tuples with variadic parameters in TS.

Make sure the return value has the exact required shape. Additionally don't resort to using Any. Also define the intersection function as well. Again, Don't use any. Should be trivial if what you say is true.

Re: Writing and linting Python at scale

#130
post #120

Earlier quoted context omitted.

Yeah they're getting flexible with the definitions. The developers are committees of people many of which don't know type theory and introduce arbitrary concepts based off of misguided intuitions. It's the same with typescript I'm sure. You will note that this thing doesn't exist in haskell for tuples because haskell devs tend to be well versed with concept of what a tuple is.

So you keep going on and on about how this is wrong, and doesn't work, and doesn't exist, and it's impossible, because it isn't implemented in Haskell? Seriously? This is one of the strangest and worst interactions I've had on this site, because I essentially made a comment that goes "the sky is blue, but it looks better at sunset when red" and you keep going "nu-uh! Impossible! Doesn't work! The sky is blue, nothing…

>So you keep going on and on about how this is wrong, and doesn't work, and doesn't exist, and it's impossible, because it isn't implemented in Haskell? Seriously?

Yeah seriously. You actually don't know what you're talking about. You're not convinced. I'm failing on that end, but this is real, you're not understanding me. Haskell one of the languages with the most advanced type systems in practical programming. Algebraic data types. You clearly haven't used it so of course you remain unconvinced.

>This is one of the strangest and worst interactions I've had on this site, because I essentially made a comment that goes "the sky is blue, but it looks better at sunset when red" and you keep going "nu-uh! Impossible! Doesn't work! The sky is blue, nothing else makes sense!"

It seems this way to you because frankly, you're not very knowledgeable of what I'm talking about. Read my other replies. I left one more to explain it to you in a bigger way. Hopefully you'll get it and stop violating the rules here with personal insults and calling this the "worst" and "strangest" interactions you ever had on this site. It's just fucking rude.

This is my last explanation for you:

https://news.ycombinator.com/item?id=38391431

If you still don't get it, well I can't help you.

Post reply on HN