Live data from Hacker News

Making Python faster with Rust

ohadravid.github.io

51–60 of 223 posts

Re: Making Python faster with Rust

#51
post #48

Using PyPy, which is a real compiler, might help. That's doing spatial data processing by exaustive search, which is inherently slow. There are better algorithms. If the number of items to be searched is large, the spatial indices of MySQL could help.

PyPy is a JIT compiler not a "real compiler", it requires warm up time to start optimizing code on runtime.

Re: Making Python faster with Rust

#53
post #51
post #48

Using PyPy, which is a real compiler, might help. That's doing spatial data processing by exaustive search, which is inherently slow. There are better algorithms. If the number of items to be searched is large, the spatial indices of MySQL could help.

PyPy is a JIT compiler not a "real compiler", it requires warm up time to start optimizing code on runtime.

In other words, PyPy is not a AOT-compiler.

Re: Making Python faster with Rust

#54
post #51
post #48

Using PyPy, which is a real compiler, might help. That's doing spatial data processing by exaustive search, which is inherently slow. There are better algorithms. If the number of items to be searched is large, the spatial indices of MySQL could help.

PyPy is a JIT compiler not a "real compiler", it requires warm up time to start optimizing code on runtime.

It does generate machine code, which CPython does not.

Re: Making Python faster with Rust

#55
post #39

I had a similar problem, when I was working as a PhD student a few years ago, where I needed to match the voxel representation of a 3D printer with the tetrahedral mesh of our rendering application. My first attempt in Python was both prohibitively slow and more complicated than necessary, because I tried to use vectorized numpy, where possible. Since this was only a small standalone script, I rewrote it in Julia in…

Numba is great if you want to write a naive loop approach in python.

Re: Making Python faster with Rust

#56
post #48

Using PyPy, which is a real compiler, might help. That's doing spatial data processing by exaustive search, which is inherently slow. There are better algorithms. If the number of items to be searched is large, the spatial indices of MySQL could help.

They tried PyPy at the beginning and it was 2x slower. Plausibly it would be better with additional optimization, but it's not cut and dry.

Re: Making Python faster with Rust

#57
post #38

The most important part of the article seems to be that this Python code is taking "an avg of 293.41ms per iteration": def find_close_polygons( polygon_subset: List[Polygon], point: np.array, max_dist: float ) -> List[Polygon]: close_polygons = [] for poly in polygon_subset: if np.linalg.norm(poly.center - point) And after replacing it with this Rust code, it is taking "an avg of 23.44ms per iteration": use pyo3::pre…

Python's for loop implementation is slow, also. You can use built in utils like map() which are "native" and can be a lot faster than a for loop with a push: https://levelup.gitconnected.com/python-performance-showdown...

Its not the looping itself that is slow in the article you linked, its that every element is appended to the list. If you use a list comprehension its even faster and it still loops over all elements of the list.

Re: Making Python faster with Rust

#59
post #50

Earlier quoted context omitted.

Today, there is a Python package for everything . The ecosystem is possibly best in class for having a library available that will do X. You cannot separate the language from the ecosystem. Being better, faster, and stronger means little if I have to write all of my own supporting libraries. Also, few scientific programmers have any notion of what C or Fortran is under the hood. Most are happy to stand on the shoulde…

>Today, there is a Python package for everything . The same could be said about CPAN and NPM. Yet Perl is basically dead and JavaScript isn't used for any machine learning tasks as far as I'm aware. WebAssembly did help bring a niche array of audio and video codecs to the ecosystem[1][2], something I'm yet to see from Python. I don't use Python, but with what little exposure I've had to it at work, its overall sluggi…

That’s a lot of opinions for so little exposure. There are a lot uses that don’t involve docker or a dozen virtual envs.

Re: Making Python faster with Rust

#60
A vectorized implementation of find_close_polygons wouldn't be very complex or hard to maintain at all, but the authors would also have to ditch their OOP class based design, and that's the real issue here. The object model doesn't lend itself to performant, vectorized numpy code.
Post reply on HN