Rust is verbose. I would use https://github.com/Suzhou-Tongyuan/jnumpy to write python extension in Julia and usually get similar performance.
Making Python faster with Rust
101–110 of 223 posts
Re: Making Python faster with Rust
#102Language design request: Take from Rust Algebraic types, ahead of time compilation and strong types, functional features, a borrow / escape checker that automatically turns shared data into Rc or Arc, as necessary, instead of tormenting me to rewrite performance irrelevant code; Take from Python the simple syntax, default pass by reference of all non-numeric types, simplified string handling, unified slice and array…
I think Nim is closest to this wishlist
Pretty much, ARC is RC without needing to manually write it. It can be as compact as Python and near C++/Rust speeds with some optimization. Then add in macros for real performance tricks like simd. Julias pretty nice as well.
Re: Making Python faster with Rust
#103Rust is verbose. I would use https://github.com/Suzhou-Tongyuan/jnumpy to write python extension in Julia and usually get similar performance.
Does it compile AoT or are you stuck with Julia start time?
Re: Making Python faster with Rust
#104This is a great article but there's still a core problem there - why should developers have to choose between accessibility and performance? So much scientific computing code suffers between core packages being split away from their core language - at what point do we stop and abandon python for languages which actually make sense? Obviously julia is the big example here, but its interest, development and ecosystem d…
IME, for having used Julia quite extensively in Academia:
- the development experience is hampered by the slow start time;
- the ecosystem is quite brittle;
- the promised performances are quite hard to actually reach, profiling only gets you so far;
- the ecosystem is pretty young, and it shows (lack of docs, small community, ...)
> what's stopping people from switching???
All of the mentioned above, inertia, perfect is the enemy of good enough, the alternatives are far away from python ecosystem & community, performances are not often a show blocker.
Re: Making Python faster with Rust
#105Earlier quoted context omitted.
The time is spent in this 3-line loop: for poly in polygon_subset: if np.linalg.norm(poly.center - point) I don't think the entire feature set of the Python runtime is involved in this.
Without using every feature you still have to conform to the complexity of the runtime. Every variable in that loop is a hash map lookup into the locals. `np.linalg.norm` is two field accesses, necessitating more hash map lookups on the module objects. `-` and `<` are attribute lookups as well as full function calls.
> Every variable in that loop is a hash map lookup into the locals.
No, it's a LOAD_FAST bytecode instruction. (The other stuff is mostly right, and probably contributes.)
Re: Making Python faster with Rust
#106This is possibly one of the best written articles end-to-end I have read. Excellent job telling the story
Agreed it was well written, but kinda pointless though since they could have “solved” the problem using the existing tools in a couple lines of code without any new deps. All that content annd profiling and they missed the fact that they were using numpy wrong.
Totally out of curiosity, could you be a bit more concrete in your posted example? I can't get it to work (I'm inexperienced with numpy and I'm messing something when translating your quick example to python)
Thanks!
Re: Making Python faster with Rust
#107Earlier quoted context omitted.
Today, there is a Python package for everything . The ecosystem is possibly best in class for having a library available that will do X. You cannot separate the language from the ecosystem. Being better, faster, and stronger means little if I have to write all of my own supporting libraries. Also, few scientific programmers have any notion of what C or Fortran is under the hood. Most are happy to stand on the shoulde…
>Today, there is a Python package for everything . The same could be said about CPAN and NPM. Yet Perl is basically dead and JavaScript isn't used for any machine learning tasks as far as I'm aware. WebAssembly did help bring a niche array of audio and video codecs to the ecosystem[1][2], something I'm yet to see from Python. I don't use Python, but with what little exposure I've had to it at work, its overall sluggi…
> WebAssembly did help bring a niche array of audio and video codecs to the ecosystem
Python already has all those: the ctypes module is just as hard to use as WebAssembly, with a much lower barrier-to-entry.
Re: Making Python faster with Rust
#108Earlier quoted context omitted.
diff --git a/poly_match_v1.py b/poly_match_v1.py index 675c88a..4293a46 100644 --- a/poly_match_v1.py +++ b/poly_match_v1.py @@ -1,4 +1,5 @@ from functools import cached_property +from itertools import compress from typing import List, Tuple import numpy as np from dataclasses import dataclass @@ -56,11 +57,8 @@ def generate_example() -> Tuple[List[Polygon], List[np.array]]: def find_close_polygons( polygon_subset: L…
I wish the author would take your suggestion (and other recommendations) here and try it out again. Would be a very interesting follow up to read: "How we speed up Python and simplifying our codebase by removing on more dependency".
Focusing on speeding up find_close_polygons instead of realizing that you're matching many points against the same set of polygons is also unfortunate, since that function being slow is a red herring. You can create a scipy.spatial.KDTree for example and just query all the points against that.
Re: Making Python faster with Rust
#109I'm wondering how much would be the speedup by rewriting the critical part in cython
Re: Making Python faster with Rust
#110Earlier quoted context omitted.
Python is a rough language to be productive in. It's a great scratchpad, but dynamic typing, exceptions/poor error handling, and a horrifying deployment and dependency system make me reach for something like Go in any case where I need something to be even vaguely reliable. The more ML I do, the more disappointed I get.
Ever tried gluing Go with either Python or JavaScript? I'm interested in learning what libraries are there to glue them and how complicated and slow they could be.
I've used gopy[0] recently to access a go library in Python. It surprisingly Just Worked, but I was disappointed by some performance issues, like converting lists to slices.