Live data from Hacker News

Writing and linting Python at scale

engineering.fb.com

91–100 of 160 posts

Re: Writing and linting Python at scale

#91

Earlier quoted context omitted.

I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education). I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only…

Python is always the second-best language for the job. Which makes it a great language to know.

Exactly. It’s not about asking if Python is the best tool. It’s about looking at the alternatives. Python comes out ahead of all alternatives, save for 1-3 others. And those others might not be feasible for a host of other reasons. That’s how you end up with Python. And it’s fine!

Re: Writing and linting Python at scale

#92
post #8

It looks like the interesting feature of their tool Fixit 2 is that its lint rules know how to auto-apply themselves. I'm fine with an auto code formatter, an auto import organizer, but not sure how much I trust a linter to auto-apply "fixes".

Adding to what other commenters said, ESLint also works like this.

However in an IDE setting it's not exactly "auto"; you have to click the light bulb and accept the fix (idk about VSCode, but in Neovim you can even get a preview of the diff [1]). This is what I'm working on a Fixit PR for right now.

[1] https://github.com/aznhe21/actions-preview.nvim

Re: Writing and linting Python at scale

#93

Earlier quoted context omitted.

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limi…

You can't even represent `Json` in Python's type system because it would require a recursive type. edit: I think this is actually a Python type annotation limitation but it's possible that's mypy, although I think in 99% of cases those are fine to conflate (I have used other Python type systems).

You can.

   from typing import Dict, List
   JSON = Dict[str, 'JSON'] | List['JSON'] | float | int | str | None
I would say the only thing ugly here is the string 'JSON' that is used for recursion. From a static safety perspective with an external type checker the static safety effect is identical.

Re: Writing and linting Python at scale

#94
post #34

Earlier quoted context omitted.

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limi…

> Python not only has types but it's type system is superior to typeScript. I strongly disagree. It might be better at some things, but it's much worse at others. Many functions can't be accurately typed (try, for example, to make a well-typed function that concatenates two arbitrary fixed-size tuples), and as far as I know generic type transformations can't be implemented (random example "this type takes a Dict[str,…

   def concat_tuples(tuple1: Tuple[int, str, float], tuple2: Tuple[str, str, str]) -> Tuple[int, str, float, str, str, str]:
       return tuple1 + tuple2

Your second function you just broke out of the type system with Any. Give me a more exact, are you saying Any is a constrained type variable? The only possibility here is this:

   T = TypeVar('T')
   def transform(x: Dict[str, T]) -> Dict[str, Callable[[], T]]:
       return {key: lambda : value for key, value in x.items()}
If you want that lambda to do something else you have to constrain T. Constraining T will give me more options in the definition:

   from numbers import Number
   def transform2(x: Dict[str, Number]) -> Dict[str, Callable[[Number], Number]]:
       return {key: lambda y: value + y for key, value in x.items()}

   def transform3(x: Dict[str, Number]) -> Dict[str, Callable[[], Number]]:
       return {key: lambda: value + value for key, value in x.items()}
etc...

The thing is because these type checkers are external, anyone can add arbitrary features to them and extend it. Someone can make it to the level of proof checking... eliminating the need for testing in general. Of course the syntax has to support it too.

Re: Writing and linting Python at scale

#95
post #82

Earlier quoted context omitted.

The tuple thing requires variadic generics from my understanding. I don't thing variadic generics support is supported in most statically typed languages. The only one I can think of right now that supports this is C++.

Typescript supports it too (quick example[0]) :) and Python actually as well, but currently you can't unpack two TypeVarTuples in the same type expression: https://peps.python.org/pep-0646/ [0] https://www.typescriptlang.org/play?#code/C4TwDgpgBAglC8UDaA...

Wow this is very cool thanks for sharing!

Re: Writing and linting Python at scale

#96
post #34

Earlier quoted context omitted.

> Python not only has types but it's type system is superior to typeScript. I strongly disagree. It might be better at some things, but it's much worse at others. Many functions can't be accurately typed (try, for example, to make a well-typed function that concatenates two arbitrary fixed-size tuples), and as far as I know generic type transformations can't be implemented (random example "this type takes a Dict[str,…

Python can almost do this with Variadic Generics from Python 3.11; its missing an exception to the single-unpacking rule (which exists to prevent ambiguity) to allow unambiguous cases. Then you would have: from typing import TypeVarTuple Ts = TypeVarTuple("Ts”) Us = TypeVarTuple("Us”) def tconcat( t1: tuple[*Ts], t2: tuple[*Us] ) -> tuple[*Ts, *Us]: ...

No this is actually wrong. It works but it doesn't capture the meaning of the nature of a tuple.

A Tuple is Typed as something as a Fixed size. That's right, it's like this at the Type level. The entire concept of a tuple is a Product type or essentially like a struct but with no names for each parameter.

   Tuple[int, str, float] #correct
Doing what you're doing here is equivalent to creating a Struct with variadic properties.

If you want some container that holds an arbitrary amount of things that is a List

The correct type for what you want is actually this:

   List[Any]

Re: Writing and linting Python at scale

#97
post #89

Earlier quoted context omitted.

Python can almost do this with Variadic Generics from Python 3.11; its missing an exception to the single-unpacking rule (which exists to prevent ambiguity) to allow unambiguous cases. Then you would have: from typing import TypeVarTuple Ts = TypeVarTuple("Ts”) Us = TypeVarTuple("Us”) def tconcat( t1: tuple[*Ts], t2: tuple[*Us] ) -> tuple[*Ts, *Us]: ...

That's the conclusion I also arrived at. The type system is slowly getting there, but many operations like this one are still not possible. It's a shame, because it makes features like decorators significantly harder to use with static typing.

I haven't seen a type system that allows variadic types for Tuples. This would be equivalent to creating a struct with variadic amount of properties.

The definition for tuples here is similar to a struct.

They are one in the same except structs have names for each property while tuples don't. That is literally the main concept of a tuple, just a struct with no names for properties.

The type system for python is already "there", it is in fact superior to many other type systems from other popular languages.

Re: Writing and linting Python at scale

#98
post #49

Earlier quoted context omitted.

For the tuple example: from typing import TypeVar T, U, V, W = TypeVar('T'), TypeVar('U'), TypeVar('V'), TypeVar('W') def concatenate(a: tuple[T, U], b: tuple[V, W]) -> tuple[T, U, V, W]: return a + b For the generic type transformation example, I'm not sure what you mean: from typing import Any, Callable Transformer = Callable[[dict[str, Any]], dict[Callable, Any]] This seems to match your question but it's really w…

Your tuple example only works if both tuples have two elements. I specifically mentioned arbitrary fixed-size tuples (as in, tuples with an arbitrary non-variable length). Your generic type transformation example also doesn't come close to what Typescript does. The resulting dict will not have known keys based on the keys of the input dict. In Typescript I can write a function that takes an object with known keys, an…

> I specifically mentioned arbitrary fixed-size tuples (as in, tuples with an arbitrary non-variable length).

This is wrong. Again, Arbitrary fixed-size tuples are equivalent to structs with an arbitrary amount of properties. Languages shouldn't do this, it destroys the nature of what a TUPLE is which is essentially just a struct with no names.

The concept you are going for is isomorphically encapsulated by ANOTHER type:

    List[Any]
You should be using the above type to encode what you want conceptually.

That being said if javascript has variadic tuples then it's not a very good type system imo. It encodes redundant concepts. Why have a tuple with Variadic arguments when I have Arrays that do the exact same thing?

Re: Writing and linting Python at scale

#99

Earlier quoted context omitted.

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limi…

Surprise: all languages have types. Superior to TypeScript is neither a high bar, nor is this any kind of objective metric. I don't know why sum types are a blessing, also I don't know why pattern matching makes anything better. I can name a lot of problems with Python, and I'm sure that libraries isn't the only one. For example, for no reason, Python has multiple unrelated mechanisms to manage program state (object,…

You seem to confuse "real"state management solutions (closures, objects) with syntactic sugar like context managers. Any class implementing __enter__ and __exit__ can be used as a context manager. The protocol doesn't impose any semantics to it. As for the presence of closures additionally to objects: this is a natural consequence of having nested functions and lexical scoping. However, it is quite uncommon to use closures to manage state in Python.

Re: Writing and linting Python at scale

#100
post #89

Earlier quoted context omitted.

That's the conclusion I also arrived at. The type system is slowly getting there, but many operations like this one are still not possible. It's a shame, because it makes features like decorators significantly harder to use with static typing.

I haven't seen a type system that allows variadic types for Tuples. This would be equivalent to creating a struct with variadic amount of properties. The definition for tuples here is similar to a struct. They are one in the same except structs have names for each property while tuples don't. That is literally the main concept of a tuple, just a struct with no names for properties. The type system for python is alrea…

That's confusing, considering Python has them with TypeVarTuples?
Post reply on HN