71–80 of 154 posts
Significant AI smell in this write up. As a result, my current reflex is to immediately stop reading. Not judgement on the actual analysis and human effort which went in. It’s just that the other context is missing.
Instead of just using a language that isn't dog slow, why not jump through these 5 different hoops? It's much easier!
Python is perfect as a "glue" language. "Inner Loops" that have to run efficiently is not where it shines, and I would write them in C or C++ and patch them with Python for access to the huge library base. This is the "two language problem" ( I would like to hear from people who extensively used Julia by the way, which claims to solve this problem, does it really ?)
It then gives you a bunch of new problems. First and foremost that you now work in a niche language with fewer packages and fewer people who can maintain the code. Then you get the huge JIT latency. And deployment issues. And lack of static tooling which Rust and Python have.
For me, as a research software engineer writing performance sensitive code, those tradeoffs are worth it. For most people, it probably isn’t. But if you’re the kind of person who cares about the Python optimization ladder, you should look into Julia. It’s how I got hooked.
The JIT work kenjin4096 describes is really promising though. If the tracing JIT in 3.15 actually sticks, a lot of this ladder just goes away for common workloads.
Earlier quoted context omitted.
While this is great, I expected faster CPython to eventually culminate into what YJIT for Ruby is. I'm not sure the current approaches they are trying will get the ecosystem there.
I implemented most of the tracing JIT frontend in Python 3.15, with help from Mark to clean up and fix my code. I also coordinated some of the community JIT optimizer effort in Python 3.15 (note: NOT the code generator/DSL/infra, that's Mark, Diego, Brandt and Savannah). So I think I'm able to answer this. I can't speak for everyone on the team, but I did try the lazy basic block versioning in YJIT in a fork of CPyth…
Missing: write static python and transpile to rust pyO3 which is at the top of the ladder. Some nuance: try transpiling to a garbage collected rust like language with fast compilation until you have millions of users. Also use a combination of neural and deterministic methods to transpile depending on the complexity.
> a garbage collected rust like language with fast compilation I don't know what languages you might have in mind. "Rust-like" in what sense?
V-lang is the one I'm tinkering with. It's like rust in terms of pattern matching as an expression, sum types, ?T instead of exceptions.
Like golang, it has shorter compile times.
I try to keep my argument abstract (that you need to lower python to something intermediate before rust) for that reason.
Earlier quoted context omitted.
> a garbage collected rust like language with fast compilation I don't know what languages you might have in mind. "Rust-like" in what sense?
It's not a popular thing to say on social media. V-lang is the one I'm tinkering with. It's like rust in terms of pattern matching as an expression, sum types, ?T instead of exceptions. Like golang, it has shorter compile times. I try to keep my argument abstract (that you need to lower python to something intermediate before rust) for that reason.
Missing: write static python and transpile to rust pyO3 which is at the top of the ladder. Some nuance: try transpiling to a garbage collected rust like language with fast compilation until you have millions of users. Also use a combination of neural and deterministic methods to transpile depending on the complexity.
One thing with python is that usually I will use one of the many c based libraries to get reasonable speed and well thought out abstractions from the start. I architect around numpy, scipy, shapely, pandas/polars or whatever. So my code runs at reasonable speed from the start. But transpiling to rust then effectively means a complete redesign of the code, data structures, algorithms etc. And I have seen the AI tools…
I wish someone writes a stdlib without using it. My attempt from a few months ago in a repo under the py2many org.