Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are you expected to run five Python type-checkers now?

pyrefly.org

Post reply on HN
Print view View original

201–210 of 220 posts

Re: Are you expected to run five Python type-checkers now?

#201

Post by SatvikBeri »

FacelessJim wrote: post #189
SatvikBeri wrote: post #24

Earlier quoted context omitted.

What statically typed language would you suggest for machine learning and large data pipelines? I don't love Python, but it has by far the largest ecosystem.

You could take a stab at Julia. It’s still dynamic in nature. But you can tune how much staticity you want. The spectrum goes from Python to C in terms of staticity. And with tools like JETLS.jl maturing you get a lot of the benefits static analysis. The data pipeline ecosystem is starting to rival that of R and Python. The fact that you can just use Julia functions while keeping the performance allows you to avoid t…

I use Julia! I like it a lot, and add type parameters to all application code. But JET.jl does not feel anywhere close to the assurances I can get from a statically typed language (yet)

Re: Are you expected to run five Python type-checkers now?

#202

Post by throwawayffffas »

HelloNurse wrote: post #177
throwawayffffas wrote: post #136

The whole type checking experience in python has disappointed me deeply and is seriously affecting my work. I see the appeal for type-checking and yeah it has caught many bugs. But the language is quickly running blindly to the worst of all worlds in regards to typing. 1. You have to exhaustively write types in many cases where they can be obviously inferred. 2. The type checking is just a lint step. i.e. we are stil…

Declaring types only where one wants to introduce a constraint would be nice, but unfortunately calls with obvious wrong types like "foo(5, {})" are a minority; in most cases types in a call depend on types of the calling function's parameters, and the type checker can only deduce unknown type from unconstrained type. Moreover, the extremely dynamic execution and lack of compilation in Python means that determining i…

I have the sneaking suspicion that for the vast majority of code it could be done.

I don't think the operator overloading is such a big issue since you can do the same kind of tracing for the overloaded operator.

The main issues would be metaclasses and all sort of monkey patching etc. i.e. anything that could mutate a type at runtime because the linter can't know if the mutation has happened.

Unfortunately I do not have the time nor patience to introduce the sixth type checker.

Re: Are you expected to run five Python type-checkers now?

#203

Post by VoidWarranty »

ActorNightly wrote: post #198
VoidWarranty wrote: post #164

Earlier quoted context omitted.

Its about limiting surface area. The gymnastics people are putting their ops teams through in order to validate oceans of generated slop is insane. Just use Rust and half of that work goes away.

Please stop trying to make Rust happen. You guys are reaching SO hard now. There is no way that Rust will be faster than simply specifying tests for an agent to run after it has generated code.

If I had said c++ would that have triggered you less?

Re: Are you expected to run five Python type-checkers now?

#204

Post by Cpoll »

pmontra wrote: post #23
dsign wrote: post #15

If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well?

Hallelujah, that's always been my position. To the static typing folks: leave my dynamically typed languages alone and go coding with something that really suit your needs. If the answer is that Python, Ruby, JS, whatever are really much more pleasant to code with, my reply is that they are so precisely because we don't have to type type definitions. Tradeoffs.

> my reply is that they are so precisely because we don't have to type type definitions.

My reply is that no, that's not why they're pleasant. If that were the only criteria, we could conclude Python is only as pleasant as Forth.

Re: Are you expected to run five Python type-checkers now?

#205

Post by chuckadams »

necovek wrote: post #199
chuckadams wrote: post #190

Earlier quoted context omitted.

I thought I'd given sufficient clues that I actually do in fact know what virtualenv is. I was hip-deep into python when it was introduced, and thought it was a clever hack, but it's relying on python being compiled to find its libraries in a path relative to the python binary, so rather than use something sensible like a launcher that sets PYTHONPATH to a local dir like every other language does, it hardlinks the py…

Usually running pip as root overcame that problem — but it did cause all the other problems virtualenvs were introduced to work around (so I am not suggesting this as the solution). You are right that there are many problems with packaging in Python, and yet I feel like virtualenvs are the smallest of those. I believe we also need to compare tech this old to tech from the same era: obviously newer ecosystems had the…

For managing dependencies in Perl, it was originally a similar story to Python: everything was system-installed, but many people would install things to their home dir and set PERL5LIB in their .bashrc. The cpan client was smart enough to detect and use the home install when writing its initial config, so you could call it a day. Later there was local::lib which fiddled @INC for the use of a project-local directory, and cpanminus defaulted to using it, and then Carton came around which is more or less a clone of Bundler from Ruby, also using local::lib under the covers.

Re: Are you expected to run five Python type-checkers now?

#206

Post by nijave »

bborud wrote: post #184
nijave wrote: post #133

Earlier quoted context omitted.

What are the problems you have with tooling? Imo it's no worse than most other languages besides a very small handful of recent ones (rust, go) where everything is included The easy approach is usually just throw it in an OCI container There's not much concrete to go on here besides "I don't like the ergonomics"

Throwing it in an OCI container is not the "easy" approach. It is the beginning of opening a can of worms. I know, because I've been developing tooling to do that at scale for the last few months and behind every layer of worms there are uglier and bigger worms that you need to deal with. And yeah, I have written a lot of code to insulate Python in containers while allowing meaningful access to hardware and services.…

Do you have examples of issues? OCI is tarball of dependencies that doesn't fight with the OS userland.

Why do you need to write code to insulate Python in containers?

At the simplest level, you can add the flags to the container runtime (network host, host ipc, host process namespace) to turn off all the namespacing besides filesystem and the Python container runs just like a non containerized process.

An extreme example

https://github.com/home-assistant/docker-base (Debian, Ubuntu, Alpine base images with Python for arm64 and amd64)

& https://github.com/home-assistant/core/blob/dev/Dockerfile (Python app built on those with >1000 deps)

And even there most of the custom code is just running a ton of combinations of inputs against docker build. The OCI container gets rid of "wide range of scenarios" for you standardizing the runtime environment

Re: Are you expected to run five Python type-checkers now?

#207

Post by HelloNurse »

throwawayffffas wrote: post #202
HelloNurse wrote: post #177

Earlier quoted context omitted.

Declaring types only where one wants to introduce a constraint would be nice, but unfortunately calls with obvious wrong types like "foo(5, {})" are a minority; in most cases types in a call depend on types of the calling function's parameters, and the type checker can only deduce unknown type from unconstrained type. Moreover, the extremely dynamic execution and lack of compilation in Python means that determining i…

I have the sneaking suspicion that for the vast majority of code it could be done. I don't think the operator overloading is such a big issue since you can do the same kind of tracing for the overloaded operator. The main issues would be metaclasses and all sort of monkey patching etc. i.e. anything that could mutate a type at runtime because the linter can't know if the mutation has happened. Unfortunately I do not…

You need to distinguish the prevention of actual type errors during the execution of a Python program, which would require mutilating the Python language, and linting Python code to find proven or likely defects, which is merely fraught with complications (open world and closed world assumptions, expecting more or less adversarial behaviour from callers of a function, etc.) and necessarily dependent on adding optional type declarations.

Re: Are you expected to run five Python type-checkers now?

#208

Post by pjmlp »

dsign wrote: post #15

If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well?

Definitely, and statically typed language have REPLs as well.

I know Python since 1.6, and it has always been mostly for OS scripting.

I do see a value on it being the new BASIC, and like BASIC, building full businesses applications with it, comes with gotchas.

Additionally, it appears the C libraries as "Python" libraries culture will never go away.

Re: Are you expected to run five Python type-checkers now?

#209

Post by pjmlp »

UltraSane wrote: post #76
dsign wrote: post #15

If you are going to be super-strict with type-checking, wouldn’t it be best to switch to a statically typed language and get the performance gains as well?

That is why I'm using C# and Rust more now than Python. You get far better RoI on types. and they are so much faster and can use all cores so much more easily.

You can even add F# into the loop, with REPL and type inference much better than those two.
Post reply on HN