Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python: The Optimization Ladder

cemrehancavdar.com

Post reply on HN
Print view View original

121–130 of 154 posts

Re: Python: The Optimization Ladder

#121

Post by pjmlp »

gcanyon wrote: post #107

People here on HN have in the past suggested that TypeScript is the superior-in-all-ways, just-as-easy/fun-to-code-in language and should replace Python in pretty much all use cases. Anyone have an opinion on how TS would fare in this comparison?

Typescript is basically a JavaScript linter.

The benefit is that JavaScript JIT compilers have a few decades of research behind them, all the way back to Smalltalk and SELF.

So in many cases you can still stay with V8 or JavaScript Core, instead of rewriting into something else, regardless of the whole rewriting into Rust that is now fashionable.

Re: Python: The Optimization Ladder

#122

Post by redgridtactical »

bee_rider wrote: post #104
redgridtactical wrote: post #75

In practice the ladder has two rungs for me. Write it in Python with numpy/scipy doing the heavy lifting, and if that's not enough, rewrite the hot path in C. The middle steps always felt like they added complexity without fully solving the problem. The JIT work kenjin4096 describes is really promising though. If the tracing JIT in 3.15 actually sticks, a lot of this ladder just goes away for common workloads.

Jax seems quite interesting even from this point of view… numpy has the same problem as blas basically, right? The limited interface. Eventually this leads to heresies like daxp b y, and where does the madness stop once you’ve allowed that sort of thing? Better to create some sort of array language.

Jax basically gives you the array language without leaving Python, and the XLA backend means you're not hand-tuning C for the GPU path. The numpy interface limitation is real though and once you need something that doesn't map cleanly to vectorized ops, you're either fighting the abstraction or dropping down anyway.

The daxpby example is a good one. Every time BLAS adds another special-case routine it's basically admitting the interface wasn't general enough. At some point you're just writing C with extra steps.

Re: Python: The Optimization Ladder

#123

Post by gf000 »

LarsDu88 wrote: post #110
scuff3d wrote: post #108

Earlier quoted context omitted.

Go and Java/C# (if you forgo all the OOP nonsense) aren't much harder to write than Python, and you get far better performance. Not all the way to Rust level, bur close enough for most things with far less complexity.

As an AI engineer I kinda wish the community had landed on Go or something in the early days. C# would also be great, although it tends to be pretty verbose. Python just has too strong network effects. In the early days it was between python and lua (anyone remember torchlua?). GoLang was very much still getting traction and in development. Theres also the strong association of golang to google, c# to microsoft, and…

Why go? That language has absolutely no expressive power, while you surely at least want to add together two vectors/matrices in AI with the + operator.

Re: Python: The Optimization Ladder

#124

Post by gf000 »

zahlman wrote: post #44
adsharma wrote: post #33

Missing: write static python and transpile to rust pyO3 which is at the top of the ladder. Some nuance: try transpiling to a garbage collected rust like language with fast compilation until you have millions of users. Also use a combination of neural and deterministic methods to transpile depending on the complexity.

> a garbage collected rust like language with fast compilation I don't know what languages you might have in mind. "Rust-like" in what sense?

Not parent, but basically every ML? OCaml, but also Scala/Kotlin to a certain degree Java, C# are all good choices.

Re: Python: The Optimization Ladder

#125

Post by gf000 »

zahlman wrote: post #82
adsharma wrote: post #78

Earlier quoted context omitted.

Here is a python AST parser written in V. It's targeting a dialect that's mostly compatible with a static subset of python3, but will break compatibility where necessary. In this case pattern matching, possibly elsewhere. https://github.com/py2many/v-ast

Never heard of this language, but it looks interesting. Very modern, certainly. One thing that stood out to me is that there's apparently the ability to write a bare `for` loop...? Is that just equivalent to while (true) in other languages?

It does have a "shady" past, but AFAIK it mostly managed to leave that vaporware behind.

Nonetheless, I would not bet anything too serious on such a small language.

Re: Python: The Optimization Ladder

#126

Post by gf000 »

adsharma wrote: post #79
tda wrote: post #38

Earlier quoted context omitted.

One thing with python is that usually I will use one of the many c based libraries to get reasonable speed and well thought out abstractions from the start. I architect around numpy, scipy, shapely, pandas/polars or whatever. So my code runs at reasonable speed from the start. But transpiling to rust then effectively means a complete redesign of the code, data structures, algorithms etc. And I have seen the AI tools…

Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive. I wish someone writes a stdlib without using it. My attempt from a few months ago in a repo under the py2many org.

> Cython and all the libs you mention use the c-api, which is the #1 thing python needs to lose to be competitive.

Quite hard to lose the #1 reason people use the language for.

Re: Python: The Optimization Ladder

#127

Post by gf000 »

threethirtytwo wrote: post #26

>The usual suspects are the GIL, interpretation, and dynamic typing. All three matter, but none of them is the real story. The real story is that Python is designed to be maximally dynamic -- you can monkey-patch methods at runtime, replace builtins, change a class's inheritance chain while instances exist -- and that design makes it fundamentally hard to optimize. ok I guess the harder question is. Why isn't python…

The more real answer is that python's primary usage is a glue language. It has to be able to interface with various C libraries, and to make the interfaces even more ergonomic, they exposed several internal details on how code is evaluated that libraries make use of (e.g. you can increment/decrement a ref counted python object's counter from C).

This pretty much makes it impossible to change many of the internal details, and to significantly optimize it.

If we remove this requirement, we get the alternative runtimes and if you check e.g. GraalPy, it has the same order of performance as JS, so your intuition is right. It's just that you have to drop supporting a good chunk of what people use Python for which is obviously a no go for most applications. (Note: GraalPy can actually also run some C libraries and in this case can cross-optimize across python and C!)

Re: Python: The Optimization Ladder

#128

Post by __rito__ »

tweakimp wrote: post #21
blt wrote: post #12

Surprised Python is only 21x slower than C for tree traversal stuff. In my experience that's one of the most painful places to use Python. But maybe that's because I use numpy automatically when simple arrays are involved, and there's no easy path for trees.

Be careful with that, numpy arrays can be slower than Python tuples for some operations. The creation is always slower and the overhead has to be worth it.

Yeah. Many seem to forget it. For one-off computation tasks, NumPy, PyTorch, JAX have non-trivial overhead, and might even be slower than vanilla Python. Only when repetition, loops, etc. come into the picture, which is recurring in many people’s workflow - JAX or NumPy is worth it.

Re: Python: The Optimization Ladder

#129

Post by adamzwasserman »

adamzwasserman wrote: post #81
seanwilson wrote: post #3

> The real story is that Python is designed to be maximally dynamic -- you can monkey-patch methods at runtime, replace builtins, change a class's inheritance chain while instances exist -- and that design makes it fundamentally hard to optimize. ... > 4 bytes of number, 24 bytes of machinery to support dynamism. a + b means: dereference two heap pointers, look up type slots, dispatch to int.__add__, allocate a new P…

The dynamism exists to support the object model. That's the actual dependency. Monkey-patching, runtime class mutation, vtable dispatch. These aren't language features people asked for. They're consequences of building everything on mutable objects with identity. Strip the object model. Keep Python. You get most of the speed back without touching a compiler, and your code gets easier to read as a side effect. I built…

Correction. I copied some incorrect values from my test harness. So Honest Python does NOT beat Dishonest Swift.

But it does beat the pants off of JS/TS on V8 which is quite the surprise.

Also in the surprise category is that Honest Java is more than 2x faster than dishonest c++.

Re: Python: The Optimization Ladder

#130

Post by gregjm »

> I don't know JAX well enough to explain exactly why it's 3x faster than NumPy on the same matrix multiplications.

JAX is basically a frontend for the XLA compiler, as you note. The secret sauce is two insights - 1) if you have enough control, you can modify the layout of tensor computations and permute them so they don’t have to match that of the input program but have a more favorable memory access pattern; 2) most things are memory bound, so XLA creates fusion kernels that combine many computations together between memory accesses. I don’t know if the Apple BLAS library has fused kernels with GEMM + some output layer, but XLA is capable of writing GEMM fusions and might pick them if they autotune faster on given input/output shapes.

> But I haven't verified that in detail. Might be time to learn.

If you set the environment variable XLA_FLAGS=--dump_hlo_to=$DIRECTORY then you’ll find out! There will be a “custom-call” op if it’s dispatching to BLAS, otherwise it will have a “dot” op in the post-optimization XLA HLO for the module. See the docs:

https://openxla.org/xla/hlo_dumps

Post reply on HN