Live data from Hacker News

Are you expected to run five Python type-checkers now?

pyrefly.org

181–190 of 220 posts

Re: Are you expected to run five Python type-checkers now?

#181
post #115

Earlier quoted context omitted.

Doesn't it come from folks that are forced to work with dynamically-typed languages but can't be arsed to understand them?

Can you elaborate?

Let me paraphrase the summary of Eloquent Ruby by Russ Olsen:

    It’s easy to write correct Ruby code, but to gain the fluency needed to
    write great Ruby code, you must go beyond syntax and absorb the “Ruby way”
    of thinking and problem solving

Re: Are you expected to run five Python type-checkers now?

#182
post #23

Earlier quoted context omitted.

Hallelujah, that's always been my position. To the static typing folks: leave my dynamically typed languages alone and go coding with something that really suit your needs. If the answer is that Python, Ruby, JS, whatever are really much more pleasant to code with, my reply is that they are so precisely because we don't have to type type definitions. Tradeoffs.

So far I have been avoiding Pydantic as a huge-ass dependency. Instead I am relying on standard type annotations, lots of typed dicts and at service/program boundaries use a jsonschema. I like being able to specify the type of most functions, and get some hints, completions and so on, but I don't want to _have to_ specify every darn type. I also don't want to write a class for everything. Typing dicts is good and usu…

I tend to get triggered when TypeScript is painted as “JS with type hints”. Coming from Python background, TS and Python with type hints are just so different.

With Python I can’t see myself type-annotating everything (or bringing in pydantic anymore for that matter, it is indeed becoming a blight), but with TypeScript my process is turned on its head: I find it natural and easy to start writing with types and have everything fully typed, and I find the fact that it simply won’t compile if anything is off (compared to Python where it’s more like “one of my N type checkers/linters failed, oh well it still runs though) a useful constraint that gives peace of mind.

Re: Are you expected to run five Python type-checkers now?

#183
post #126
post #71

Earlier quoted context omitted.

Well, that's the curse of machine learning: since everyone uses Python you have to deal with Python. Even though Python isn't very nice when things start to get serious and you don't want to spend your time fiddling with noise just to make something work at scale. I'd wish the ML/AI/LLM crowd would see that it is in their interest to get better developer ergonomics at scale. (I don't want to have to turn to C++)

Anything performance sensitive ends up being an extension in compiled code anyway Python is mostly just glue

The performance is not the (only) issue. The issue is the death by a thousand cuts involved in distributing Python programs without a two page set of instructions that have to be followed to make it work. It is rarely "just works" unless you can make a lot of assumptions about the environment it runs in. It's why I generally steer away from any application written in Python. It is going to be painful.

However my experience might be a bit different since I actually have to deal with Python at scale and in a fairly dynamic environment.

I need to run hundreds of Python programs, written by dozens of programmers, over many years, that speaks to custom hardware, runs on a remote site, in a production environment that has to work and with new versions of things coming in all the time. Some of these Python programs not only link with C libraries, but run external binaries because developers didn't have time to integrate them as libraries because it takes forever to make it work on all the different os/arch combinations (easier to just run the C code in subprocesses).

This runs on three different CPU architectures (we're trying to eliminate one of them), two different operating systems and a pretty wide mix of hardware and system configurations I need to insulate Python from. Much of the hardware being custom built stuff. Because Python has a lot of exposed surface to the OS compared to a statically linked binary. (Roughly 100x the surface of statically linked binaries that don't link with libc -- which is evident by the insanely bloated OCI images that result from packaging what you need to run)

Modern compiled languages that have sorted toolchains makes it pretty easy to produce "production grade" os/arch specific binaries that can survive almost everywhere. You compile build a statically linked binary for each architecture to overcome the challenges of varied Linux runtime environments (see Linus T's frustrations with Linux and software distribution - it's not like it is easy to begin with). Go and Rust do this well.

So you end up having to containerize everything in ephemeral containers to lock down the execution environment while retaining some speed. But of course it isn't that simple, because if you depend on access to weird hardware and/or you run on custom built machines you have to detect this and ensure the application inside the container gets access to the things it needs from the container. So you have to fix that.

In a way that is almost completely invisible to the developer.

All of this has to be understandable and _reduce_ complexity for developers and operators so at the very least you don't follow the Python philosophy of "just throw another layer of complexity on it and make the instructions another page longer".

40-50kLOC later (in Go and Python, I have lost count) of code to try to make the problems go away, and I have something that is on the verge of actually being usable in a production environment for taming wayward Python code.

The easiest fix? If people could stop using Python because they don't want to learn a language that can produce something that is easier to distribute to users.

Believe me, I have spent months now trying to make Python work properly in a challenging environment. The only way this "worked" before was by just lowering standards to where the definition of "works" is flexible enough to count daily dumpster fires as "nominal". And of course people don't care. Python fosters a "it works for me" mentality where people don't know and don't care what it is like to be on the receiving end.

90% of problems I have because of Python would just disappear if people used languages that can produce robust binaries with limited exposure to system peculiarities. But that kind of requires people to understand why it is a problem in the first place. And people generally don't bother to know.

Re: Are you expected to run five Python type-checkers now?

#184
post #133
post #69

Earlier quoted context omitted.

Python has other, bigger problems that make it a constant headache. One of them being the dismissive attitude towards any and all of problems that come from versioning, dependencies and quirks that make it challenging to have robustness. Criticisms are typically dismissed by suggesting heaping yet another "solution" onto the growing pile of "solutions" that you have to drag around with you. That people have to learn.…

What are the problems you have with tooling? Imo it's no worse than most other languages besides a very small handful of recent ones (rust, go) where everything is included The easy approach is usually just throw it in an OCI container There's not much concrete to go on here besides "I don't like the ergonomics"

Throwing it in an OCI container is not the "easy" approach. It is the beginning of opening a can of worms. I know, because I've been developing tooling to do that at scale for the last few months and behind every layer of worms there are uglier and bigger worms that you need to deal with.

And yeah, I have written a lot of code to insulate Python in containers while allowing meaningful access to hardware and services. While at the same time not heaping more complexity and cognitive surface at the developer. Including writing my own container software to actually understand what's involved at a more detailed level so I know what I'm doing when trying to make this work with existing container software. (No, I don't run any of my container managers in production since I don't want to maintain it -- but this also means a bit more complexity in using existing ones)

It may be "easy" in trivial cases, but it is very far from easy if you want to make something that can cater to a wide range of scenarios.

Re: Are you expected to run five Python type-checkers now?

#186
post #23

Earlier quoted context omitted.

Hallelujah, that's always been my position. To the static typing folks: leave my dynamically typed languages alone and go coding with something that really suit your needs. If the answer is that Python, Ruby, JS, whatever are really much more pleasant to code with, my reply is that they are so precisely because we don't have to type type definitions. Tradeoffs.

Totally agree. I hear a lot of rust makes it hard to write incorrect programs. In my experience it makes it hard to write programs in general.

[deleted]

Re: Are you expected to run five Python type-checkers now?

#187

The whole type checking experience in python has disappointed me deeply and is seriously affecting my work. I see the appeal for type-checking and yeah it has caught many bugs. But the language is quickly running blindly to the worst of all worlds in regards to typing. 1. You have to exhaustively write types in many cases where they can be obviously inferred. 2. The type checking is just a lint step. i.e. we are stil…

Ocaml is probably closest to that. The ML language family has roots in generic code with global type inference.

Re: Are you expected to run five Python type-checkers now?

#188

Earlier quoted context omitted.

Yes, where would I be without the _RelationshipBackPopulatesArgument type of sqlalchemy.orm.relationship(argument: _RelationshipArgumentType[Any] | None = None, secondary: _RelationshipSecondaryArgument | None = None, *, uselist: bool | None = None, collection_class: Type[Collection[Any]] | Callable[[], Collection[Any]] | None = None, primaryjoin: _RelationshipJoinConditionArgument | None = None, secondaryjoin: _Rela…

Part of it is due to the clunky `_NoArg.NO_ARG` business for optional params. Pretty-printing it would also go a long way, but that technology seems too advanced for any language circa 2026.

This is a big part of the reason that I've embraced ths sqlc (d/re)evolution.

Writing queries in sql and then generating for the target language also provides a flexibility that has reduced rewrite cost. Add to this ease of organization and layoit, and I'm not going back.

Re: Are you expected to run five Python type-checkers now?

#189
post #15

If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well?

What statically typed language would you suggest for machine learning and large data pipelines? I don't love Python, but it has by far the largest ecosystem.

You could take a stab at Julia.

It’s still dynamic in nature. But you can tune how much staticity you want. The spectrum goes from Python to C in terms of staticity. And with tools like JETLS.jl maturing you get a lot of the benefits static analysis.

The data pipeline ecosystem is starting to rival that of R and Python. The fact that you can just use Julia functions while keeping the performance allows you to avoid those weird vectorization gymnastics. The ML ecosystem is also in a great state. JUMP.jl, Touring.jl, the whole SciML ecosystem, autodiffing and gpu computing are all close to best in class in terms of quality. The NN side of ML is a a bit weaker, but just for lack of developer time investment into that side of the ecosystem.

Re: Are you expected to run five Python type-checkers now?

#190

Earlier quoted context omitted.

Python the language is pretty nice. It has its warts, but I make my living in PHP which is practically made of warts. But the python ecosystem still seems to be trying to figure out this whole package management and project setup thing. In most languages I can do some form of `$blub install` where $blub is the language's official package manager or some close equivalent. It's just python that always screams at me tha…

I am not sure you get what virtualenvs are: Python is never screaming at you to set up a virtualenv, it must be a particular package recommending use of virtualenv for easy set up without interfering with the rest of your system. Virtualenv allows you to seamlessly run multiple Python ecosystems simultaneously, even within the same project directory. It's basically primitive containerisation mechanism that predates a…

I thought I'd given sufficient clues that I actually do in fact know what virtualenv is. I was hip-deep into python when it was introduced, and thought it was a clever hack, but it's relying on python being compiled to find its libraries in a path relative to the python binary, so rather than use something sensible like a launcher that sets PYTHONPATH to a local dir like every other language does, it hardlinks the python install into the local dir in order to pretend everything is a system-wide global install. I always figured that hack was a temporary workaround, but it's going on a couple decades now.

As for the screaming, I'm talking about pip, which has the unique property of just refusing to work by default unless you use --user (what most other package managers now call "global") or are in one of these pretend-global environments. Ultimately it's just a paper cut, not a show-stopper, but it points to not just a failure to address the pain points of the package ecosystem, but a seeming refusal to do so.

Post reply on HN