Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing and linting Python at scale

engineering.fb.com

Post reply on HN
Print view View original

141–150 of 160 posts

Re: Writing and linting Python at scale

#141

Post by dragonwriter »

corethree wrote: post #126
dragonwriter wrote: post #106

Earlier quoted context omitted.

> It works but it doesn't capture the meaning of the nature of a tuple. That's backwards. It doesn't work (as noted in GP, because Python supports unpacking only a single variadic parameter in a type annotation), but it does capture the nature of a tuple. It is (or, more precisely, would be , if the syntactic limitation was relaxed to allow it) a generic function that operates on two tuples of arbitrary tuple types a…

> a generic function that operates on two tuples of arbitrary tuple types and returns a tuple of a third tuple type whose shape is a simple concatenation of the shapes of the input tuples. This isn't definable without dependent types or a concat operator at the type level. Dependent types aren't a thing we are talking about here and languages that support this feature are waaay out of scope. You're basically asking f…

> This isn't definable without dependent types or a concat operator at the type level.

It takes relaxing one overeaeger limitation in Python's type annotation syntax, a limitation which is necessary in general to avoid ambiguity, but could be relaxed in specific, enumerable cases that do not produce ambiguity.

> You're basically asking for stuff like this:

I'm not sure exactly what your examples mean because I'm not sure exactly what syntax that is supposed to be, but in actual Python type annotations with a relaxation of the no multiple unpacking rule for variadic generics, what would be allowed is things like the example I posted upthread for tuples:

  def tconcat(t1: tuple[*Ts], t2: tuple[*Us]) -> tuple[*Ts, *Us]: ...
or, for Arrays:

  def product(a1: Array[DType1, *Shape1], a2: Array[DType2, *Shape2]) -> Array[tuple[DType1, DType2], *Shape1, *Shape2]: ...
concat for arrays would take dependent types, but that's not what is being discussed.

> This doesn't exist in the realm of programming we're talking about.

All the concepts necessary for what the upthread poster noted TypeScript already has and was superior to Python because Python lacks, and which I pointed out Python is relaxing one limitation short of also having, already exist in Python since Python 3.11, there is just a syntactic limitation on using the necessary combination to avoid ambiguities (which occur in different circumstances). And, of course, all the necessary concepts also exist in TypeScript. And those two languages are "the realm of programming we're talking about".

> You want something well out of scope of this conversation

No, literally, I'm discussing the thing that is the advantage of TypeScript's type system over Python's that has been the subject of this conversation, which is not, ipso facto, well out of the scope of the conversation.

> and well out of scope of most practical programming languages.

It exists in TypeScript which is widely used, whether or not you thin it is "practical".

> These operators only come into play with advanced theoretical languages like Lean or Idris and it's very off topic.

The discussion was about a comparison between TypeScript's type system (which supports this) and Python's (which does not quite, though it has all the pieces with a syntactic restriction that prevents certain useful combinations.) These are not "advanced theoretical languages", and the discussion you jumped into is not at all off-topic (certainly not of itself, despite your jumping in to derail it, and not of the broader discussion of Python and scale because JS/TS is a similar industrially popular language ecosystem to Python with a dynamically typed base and optional statically type-checked layer.

Re: Writing and linting Python at scale

#142

Post by corethree »

Timon3 wrote: post #133
corethree wrote: post #130

Earlier quoted context omitted.

>So you keep going on and on about how this is wrong, and doesn't work, and doesn't exist, and it's impossible, because it isn't implemented in Haskell? Seriously? Yeah seriously. You actually don't know what you're talking about. You're not convinced. I'm failing on that end, but this is real, you're not understanding me. Haskell one of the languages with the most advanced type systems in practical programming. Alge…

> Haskell one of the languages with the most advanced type systems in practical programming. Algebraic data types. You clearly haven't used it so of course you remain unconvinced. Wrong. > It seems this way to you because frankly, you're not very knowledgeable of what I'm talking about. This wouldn't come across quite as bad if I didn't literally post an example long ago. If you're so knowledgeable about all this, wh…

I'm not, I acknowledge everything you say. It may seem that I'm not acknowledging stuff. But this is not the case. I read and understand all your posts.

You on the other hand. I doubt you read or understood all my responses to you. You only partially understand.

I also don't think you have used haskell before. This is what I mean. You aren't knowledgeable.

Re: Writing and linting Python at scale

#143

Post by corethree »

Timon3 wrote: post #131
corethree wrote: post #127

Earlier quoted context omitted.

>My god. No, Python doesn't support "fixed size arrays", but it does support tuples of arbitrary size, no matter how often you claim that to be wrong. My god I never claimed this. I'm just talking about programming in general. I'm trying to show you that what YOU are asking for is programming operations to happen at the type level. You're not seeing it correctly. >I could, for example, do the thing I've been writing…

The operator exists. I can use it today, without exiting the type system, both in Python (limited to only unpacking one tuple type) and Typescript (not limited). Why do you keep claiming this to be impossible, when it already exists right now? It's also in no way a special case in TS, the type system supports many things you need dependent types for. Maybe read up on it before you claim it can't do these things?

Great. Define the concat and intersection function. Do it in python and ts. That is your claim prove it.

>Maybe read up on it before you claim it can't do these things?

Maybe don't be rude. If I'm wrong I'm wrong. But if you say something like this you need to back up your words. You "read" it. So prove it.

Re: Writing and linting Python at scale

#144

Post by corethree »

dragonwriter wrote: post #140
corethree wrote: post #128

Earlier quoted context omitted.

Bro, Look at this: https://peps.python.org/pep-0646/#multiple-type-variable-tup... Does this not exactly mean: "To make it arbitrary size you have to constrain it on a single generic parameter"

> Bro, Look at this: I’m not your bro and I've been discussing what that PEP does and does not allow in every post in this subthread. > Does this not exactly mean: "To make it arbitrary size you have to constrain it on a single generic parameter" No, it doesn't: Ts = TypeVarTuple(*Ts) def f(cond: Callable[[],bool], t: tuple[*Ts]) -> None | tuple[*Ts]: ... The parameter t is an arbitrary-sized (and shaped) tuple, and…

>I’m not your bro and I've been discussing what that PEP does and does not allow in every post in this subthread.

It's called an expression. It's a common one in the US and in countries that have English as a first language.

>No, it doesn't:

Lol yes it does. I said what I said and meant what I meant. I'm in control of that you aren't. What's going on here is that you either misinterpreted it or you're just wrong.

>No multiple unpacking means you can't type things like concatenations of arbitrary-shaped tuples without losing type specificity (and resorting to just tuple), but it doesn't mean you can't type functions using arbitrary sized and shaped tuples outside of that restriction.

You literally and I mean literally re-explained what I meant. Then Proceeded to imply I meant something else. Read it: to make a parameter of arbitrary type you have to constrain it on a single parameter.

Word for word. Don't twist it.

Re: Writing and linting Python at scale

#145

Post by Timon3 »

corethree wrote: post #143
Timon3 wrote: post #131

Earlier quoted context omitted.

The operator exists. I can use it today, without exiting the type system, both in Python (limited to only unpacking one tuple type) and Typescript (not limited). Why do you keep claiming this to be impossible, when it already exists right now? It's also in no way a special case in TS, the type system supports many things you need dependent types for. Maybe read up on it before you claim it can't do these things?

Great. Define the concat and intersection function. Do it in python and ts. That is your claim prove it. >Maybe read up on it before you claim it can't do these things? Maybe don't be rude. If I'm wrong I'm wrong. But if you say something like this you need to back up your words. You "read" it. So prove it.

I defined the concat function in TS and Python below: https://news.ycombinator.com/item?id=38391702

Now, obviously Python doesn't support it due to a restriction in the TypeVarTuple PEP, which is the literal entire point of everything I have written. But what more are you asking for? What more do you want as an example?

I am not going to touch the intersection function, as I haven't made any claims in that direction.

Re: Writing and linting Python at scale

#146

Post by corethree »

Timon3 wrote: post #132
corethree wrote: post #129

Earlier quoted context omitted.

>Yes, and if Python had a better type system, I could write a more concrete definition. Why don't you understand that? I understand everything. It's you who doesn't understand. What I described is for ALL type systems in general. Not python types. >Of course it's possible, the type system even supports almost everything that's necessary. You can already define what I want with a single fixed-size tuple of arbitrary s…

> I understand everything. It's you who doesn't understand. What I described is for ALL type systems in general. Not python types. Then why is what you're claiming to be impossible possible in Typescript? How can that be? Is the whole world wrong, aside from you? > Look just prove it to me. Define the type signature for concat function that will concat two tuples with variadic parameters in TS. You mean the one I alr…

Let me address the one in python.

First off we are talking about concatenating tuples.

What you're doing here is called preppending an element to a tuple. Like come on man. Have you not been reading the examples I've been posting.

Your typescript example is relevant. But like I said it's an arbitrarily narrow thing. It's a special case. Typescript arbitrarily only supports only concat. Which is what I've been saying. It doesn't type level programming.

>Why does Python disagree with you?

There's no need to be a complete ass hole. Why does not being an ass hole disagree with you? First off your python example is completely wrong. Second off no point in saying snarky comments like that.

Re: Writing and linting Python at scale

#147

Post by Timon3 »

corethree wrote: post #142
Timon3 wrote: post #133

Earlier quoted context omitted.

> Haskell one of the languages with the most advanced type systems in practical programming. Algebraic data types. You clearly haven't used it so of course you remain unconvinced. Wrong. > It seems this way to you because frankly, you're not very knowledgeable of what I'm talking about. This wouldn't come across quite as bad if I didn't literally post an example long ago. If you're so knowledgeable about all this, wh…

I'm not, I acknowledge everything you say. It may seem that I'm not acknowledging stuff. But this is not the case. I read and understand all your posts. You on the other hand. I doubt you read or understood all my responses to you. You only partially understand. I also don't think you have used haskell before. This is what I mean. You aren't knowledgeable.

> I'm not, I acknowledge everything you say. It may seem that I'm not acknowledging stuff. But this is not the case. I read and understand all your posts.

Then why have you not yet acknowledged the Typescript concat function I posted here yesterday? You have since written 16 replies, not a single one of which acknowledges this. Why do you keep not acknowledging that I gave you the function you consider impossible?

Re: Writing and linting Python at scale

#148

Post by Timon3 »

corethree wrote: post #146
Timon3 wrote: post #132

Earlier quoted context omitted.

> I understand everything. It's you who doesn't understand. What I described is for ALL type systems in general. Not python types. Then why is what you're claiming to be impossible possible in Typescript? How can that be? Is the whole world wrong, aside from you? > Look just prove it to me. Define the type signature for concat function that will concat two tuples with variadic parameters in TS. You mean the one I alr…

Let me address the one in python. First off we are talking about concatenating tuples. What you're doing here is called preppending an element to a tuple. Like come on man. Have you not been reading the examples I've been posting. Your typescript example is relevant. But like I said it's an arbitrarily narrow thing. It's a special case. Typescript arbitrarily only supports only concat. Which is what I've been saying.…

> Let me address the one in python.

> First off we are talking about concatenating tuples.

> What you're doing here is called preppending an element to a tuple. Like come on man. Have you not been reading the examples I've been posting.

I have no idea how you still do not understand what I am talking about.

My whole point is that Python doesn't support this. It has all the ingredients, it literally supports what you called impossible (having a function return a tuple type that's based on an arbitrary fixed-size input). Yes, the function doesn't support concatenating two tuples while being well-typed. That is what every single one of my messages was about. How do you read all my messages and not understand my basic argument?

And, please explain in great detail: if a typing system can handle prepending/appending an element to a tuple, what is the big difference to concatenating two fixed-size tuples? Why is one obviously possible, and one isn't? What is the fundamental difference?

> Your typescript example is relevant. But like I said it's an arbitrarily narrow thing. It's a special case. Typescript arbitrarily only supports only concat. Which is what I've been saying. It doesn't type level programming.

First off, you're wrong, it's not an arbitrarily narrow thing - Typescript does allow much more complex operations, the type system is literally Turing-complete.

But aside from that: I didn't say that it supports anything else. All I have been saying is: Typescript supports this thing, which Python does not. Why do you keep disagreeing and saying it's "wrong" and "impossible" when it does literally do what I've been telling you?

> There's no need to be a complete ass hole. Why does not being an ass hole disagree with you? First off your python example is completely wrong. Second off no point in saying snarky comments like that.

I have spent enough time trying to explain this to you. You're still fundamentally misunderstanding what I am talking about while acting like you know everything. You're either unable to understand it or a troll - it doesn't matter which it is.

Re: Writing and linting Python at scale

#149

Post by corethree »

dragonwriter wrote: post #141
corethree wrote: post #126

Earlier quoted context omitted.

> a generic function that operates on two tuples of arbitrary tuple types and returns a tuple of a third tuple type whose shape is a simple concatenation of the shapes of the input tuples. This isn't definable without dependent types or a concat operator at the type level. Dependent types aren't a thing we are talking about here and languages that support this feature are waaay out of scope. You're basically asking f…

> This isn't definable without dependent types or a concat operator at the type level. It takes relaxing one overeaeger limitation in Python's type annotation syntax, a limitation which is necessary in general to avoid ambiguity, but could be relaxed in specific, enumerable cases that do not produce ambiguity. > You're basically asking for stuff like this: I'm not sure exactly what your examples mean because I'm not…

I see. So ts arbitrarily supports concatenation of generic parameters via unrolling.

All right. I can admit TS is superior from this angle because of that arbitrary support. And I can admit that the arbitrary support follows from the way both languages use unrolling syntax.

Re: Writing and linting Python at scale

#150

Post by _whiteCaps_ »

kstrauser wrote: post #29
_whiteCaps_ wrote: post #7

There's a lot of neat Python research coming from Facebook. libcst (which Fixit uses) is super cool - I use it in https://gitlab.com/harford/logzy

Why do you use that instead of flake8, which also finds that? (Not meant to be a pointed question; genuinely curious.)

AFAIK flake8 only lints and doesn't apply the fixes.

fixit probably does a better job than my tool but it was fun to write. :-)

Post reply on HN