Earlier quoted context omitted.
The fact that you even didn't mention webpack, once a champion, is especially sad.
Doesn't Next.js still use webpack?
It's turbopack time: https://nextjs.org/blog/next-16-beta
Pyrefly: Python type checker and language server in Rust
121–130 of 150 posts
Re: Pyrefly: Python type checker and language server in Rust
#122I've tried pyrefly, ty, pyright, and basedpyright, with a large complex code base written using PyCharm, and _none_ of them do as good a job as PyCharm, particularly in discovering more complex type inheritances. It's a pity because in other respects Zed (which relies on these) is a worthy competitor to PyCharm (and much faster!) -- but the endless squiggly lines because pyrefly can't figure out the type, is annoying…
That's interesting. In my experience basedpyright has been better than pyCharm at every turn. Yes, a lot more warnings but generally sensible. One specific thing I remember pissing me off: https://youtrack.jetbrains.com/issue/PY-58665 Still open 2 years later.
Re: Pyrefly: Python type checker and language server in Rust
#123Earlier quoted context omitted.
I get why they'd panic, but why have enough segfaults that you noticed? So, I went and read the code. Zuban seems to have a bunch of scary "I'm not sure if this is correct" unsafe blocks, which to me would be a red flag. I mean, it's better that there's a comment expressing the doubt, but my experience is that if you're not sure whether it's correct, it's probably not correct.
While acknowledging the risk of causing a pile-on (which I don't want!), would it be possible to have a link to them or a description of what the unsafe blocks accomplish? I'm intrigued if they are for performance or API ergonomics, if they are due to limitations of the borrow checker, the stdlib or crate dependencies. For anyone reaching for unsafe, there are in many cases either an existing API (split_at_mut comes…
I skimmed these, so this is nothing close to a survey, much less a comprehensive review of the software, however
For example in crates/zuban_python/src/file/diagnostics.rs:
"TODO this unsafe feels very wrong, because a bit lower we might modify the complex/ points."
or crates/zuban_python/src/database.rs:
"Points are guarded by specific logic and if they are overwritten by something that shouldn't it should not be that tragic."
I saw nothing where I was like "ZOMG this is definitely busted" but I definitely did not get the robust "Oh, I see now why this is correct" that I like from a good unsafe rationale comment, and these aren't tiny things like the small unsafe bit twiddling transmutes which are probably either actually correct or in any case will do what you expected at compile time and so any surprises are priced in without a rationale text.
Re: Pyrefly: Python type checker and language server in Rust
#124Python is starting to feel a bit like JavaScript circa 2014. Remember grunt, gulp, webpack, coffeescript, babel? Now we've got pyright, mypy, pyrefly, black, ruff, ty, flake8, poetry, uv... I used to find this kind of tooling explosion exhausting (and back then with JS it truly was), but generally speaking it's a good sign that the community is hungry to push things forward. Choice is good, as long as we keep the Uni…
Is there any meaningful difference between TS and Python?
Re: Pyrefly: Python type checker and language server in Rust
#125Earlier quoted context omitted.
Python's type system is the problem here, which cannot support even the native dataclass pattern and needs custom plugins written for each type checker.
I don't think this is really true, I think native data classes can do just about everything you need, it's just that you need some tooling around them to get the same as Pydantic offers. I'm pretty sure we are going to see better native dataclass support in Pydantic going forward. Already there is some support in Pydantic for native dataclasses: https://docs.pydantic.dev/latest/concepts/dataclasses/
There is no facility in the type system to express the idea that "whatever class attributes are defined, those are kwargs to the init function". That's hacked in using extra typechecker plugins. That's why we're sitting around talking about if a particular typechecker "supports pydantic", rather than it just working.
Re: Pyrefly: Python type checker and language server in Rust
#126Earlier quoted context omitted.
Python's type system is the problem here, which cannot support even the native dataclass pattern and needs custom plugins written for each type checker.
The type system’s alright. It just gets especially tricky when you’re trying to check code which won’t exist until you run it. For instance, suppose you wanted to load your own module from a database with something like foo = eval(result) It can’t know what you’re going to load until it actually does it. Things which lean heavily into metaprogramming, typically ORMs or things like Pydantic, fall into that category. I…
> I can’t hold that against the type system.
I think we should. Dataclasses have existed in Python for an extremely long time, and yet the type system doesn't support defining your own similar classes. Kwargs have also existed forever, but they forgot to support that and had to add TypedDict's much later. And it still doesn't properly support optional fields. There's a lot of stuff like this in the language which are unbelievably frustrating, because for some reason they implemented the syntax before implementing a typechecker. Everything has been hacked in ever since. I consider python's type system to be a lost cause, just hoping for someone to make the Typescript equivalent for Python.
Re: Pyrefly: Python type checker and language server in Rust
#127Earlier quoted context omitted.
> New typecheckers don't need to be perfect. They need to be good enough, easy to integrate and have low false positives. Sure, they will improve with time, but if feels like a pain then no one will pick it up. Python users are famously averse to tools that slow down their dev cycles, even if it means better long term stability I really don't agree. Sure, they don't need to be perfect but also keep in mind many codeb…
No, I'm pretty sure they need to be perfect. If the tool's telling me bad information about types, first I'm going to lose a stupid amount of time debugging to that wrong info, and then swear off the tool forever after I realize it was the tool that was wrong.
Type checking in Python involves guessing. Take a program like
x = [3]
Should the type checker guess that the type of x is list[Any], or list[int], or list[Literal[3]], or something else? Libraries you depend on will pose more difficult versions of this question.
So it's not possible to expect the tool to be "perfect", whatever that means - usually people think of it as meaning it allows all code that they think is idiomatic and reasonable, and disallows all code that could raise a TypeError at runtime. You can only expect the tool to be reasonably good and perhaps to have an opinionated design that matches the way you would like to write Python.
Re: Pyrefly: Python type checker and language server in Rust
#128Earlier quoted context omitted.
I don't think this is really true, I think native data classes can do just about everything you need, it's just that you need some tooling around them to get the same as Pydantic offers. I'm pretty sure we are going to see better native dataclass support in Pydantic going forward. Already there is some support in Pydantic for native dataclasses: https://docs.pydantic.dev/latest/concepts/dataclasses/
There is no facility in the type system to express the idea that "whatever class attributes are defined, those are kwargs to the init function". That's hacked in using extra typechecker plugins. That's why we're sitting around talking about if a particular typechecker "supports pydantic", rather than it just working.
There is a way to indicate whatever class attributes are defined, those are kwargs to the init function. For one, you can use `kw_only` on the dataclass decorator [1], alternatively you can use the `kw_only` arg on the field function [2].
[1]: https://docs.python.org/3/library/dataclasses.html#dataclass...
[2]: https://docs.python.org/3/library/dataclasses.html#dataclass...
Re: Pyrefly: Python type checker and language server in Rust
#129Sadly, the question with both this and ty is: Does it support pydantic? If not, then it's not really helpful for many people, and right now AFAIK neither supports pydantic. Pydantic is probably the problem here, but it is what it is.
Why would they need to support pydantic when it uses standard annotations? I tried it with ty yesterday and it flagged issues as expected.
Pydantic does some pretty strange stuff which works as expected but confuses the type checker unless you use the pydantic plugin. https://docs.pydantic.dev/latest/integrations/mypy/
Re: Pyrefly: Python type checker and language server in Rust
#130Earlier quoted context omitted.
No, I'm pretty sure they need to be perfect. If the tool's telling me bad information about types, first I'm going to lose a stupid amount of time debugging to that wrong info, and then swear off the tool forever after I realize it was the tool that was wrong.
Type checking in Python involves guessing. Take a program like x = [3] Should the type checker guess that the type of x is list[Any], or list[int], or list[Literal[3]], or something else? Libraries you depend on will pose more difficult versions of this question. So it's not possible to expect the tool to be "perfect", whatever that means - usually people think of it as meaning it allows all code that they think is i…
Arguably it's the job of a linter and not a type checker, but if your code just assigns random things to random variables that never get used again, I hope something would point that out to you before it passes code review and merged to main,
ideally before it even gets sent for review.
Ignoring the question of why there's some random assignment to x in the first place, where are its type hints? Those were added starting with Python 3.5 via PEP 484 just over 10 years ago and have been added to since then. If our goal is maintainable code at scale, the first thing I'd expect of a type checker is for it to communicate with the user, stating that a) the variable in file foo.py on line 43 is missing a type hint and couldn't be guessed with confidence, and that it's just gonna guess it's a list[int]. For bonus points, the tool could run in interactive mode and ask the user directly. But even without a hint, assuming the variable actually is used later, how does it get used?
It seems like you'd just start with the most strict interpretation, list[Literal[3]], and relax it as it gets used. If it gets fed into a function that doesn't have hints and can't be introspected for whatever reason, relax it to list[int] and then further relax it to list[Any] as necessary. Then depending on the mode the user configured the tool to run in, print nothing, or a warning, or error out or ask the user what to do if running interactively. Some of the config options could be strict, tolerant, and loose. Or maybe enforcing, permissive, and loose. Whatever color we want this bike shed to be.
More advanced tools let the user configure individual issues and their corresponding levels, but that may be considered too many options to give the user, overwhelming the user who then doesn't use the tool.
As far as perfect typing goes, I mean, yeah, ideally, after satisfying the type checker in strict mode, which means no types need to be guessed at, as all variables and other locations without type annotations that need them were reported, or even annotated interactively, or if we're real fancy, automatically with a comment. IMO programs should feel free to edit code directly (as a non-default option) That would mean the program could not throw an an uncaught TypeError, unexpectedly. That's a lot to ask of a dynamically strongly typed language, but I'd settle for it never being utterly wrong.
What that means is if the type checker says the function is returning a string, but of the 200 calls to that function, the type checker throws an error and says ten of those callsites expect it to return a dict, all I'm saying is this hypothetical type checker had better be right about that. Nothing worse than losing hours rooting around in the code looking for something that isn't actually there (or an errant semicolon, but this isn't C)