51–60 of 74 posts
Coming from someone who uses python but doesn't really follow alternative compilers, PyPy sounds great. What are some of the downsides, if any? Are you sacrificing library compatibility for faster core+standard libs?
Why are they still comparing to Python 2.7.2? I couldn't find benchmarks against Python 3.5 for their Py3 interpreter. All the times I tried PyPy I came into a hurdle where one of the libraries I needs doesn't work (or underperforms) in PyPy, the most important ones being Numpy and OpenCV. So in the end I just gave up with them, and stuck with Python 2/3 and Cython, which solved my speed problems without having to do…
The next benchmark "only" runs 6x slower in PyPy ; still bad, but that paints quite a differen picture.
Why are they still comparing to Python 2.7.2? I couldn't find benchmarks against Python 3.5 for their Py3 interpreter. All the times I tried PyPy I came into a hurdle where one of the libraries I needs doesn't work (or underperforms) in PyPy, the most important ones being Numpy and OpenCV. So in the end I just gave up with them, and stuck with Python 2/3 and Cython, which solved my speed problems without having to do…
The benchmark showing a factor 100x compares a pure python implementation of "pidigits" running in PyPy3 vs one that uses GMP (via gmpy2) in python. I am actually impressed it's only a factor 100. And with progress made by cffi, I'd hopeful that a GMP-using PyPy version could be written rhat matches the speed of the code using gmpy2. The next benchmark "only" runs 6x slower in PyPy ; still bad, but that paints quite…
For the record, I'm not claiming PyPy3 to be 100x slower, these are benchmarks and they're hairy beasts that we have to shave and try to see what they tell us about performance.
My point is that PyPy3 is still behind in language features (in CPython we have a lot of nice things from 3.6 and 3.7 coming soon, while PyPy still lags behind having the complete 3.5 feature set), and they haven't optimized it as much as their 2.7 branch. But the PyPy people are always showing those "7x faster than CPython" claims (which come from an average of benchmarks, which seems to have been cherry picked to avoid the ones in which they're actually slower than CPython).
On the other hand, with Cython moving over to Py3 was never an issue (it actually helped in some cases), and it always helped to deliver better performance, and in just the right spot where it's needed. True, you have to know about profiling and identifying where to use Cython, but at my workplace it's been a far better tool to solve our performance needs.
I have a question and then a general vent 1. Does anyone know the latest update on NumPyPy? PyPy for me is just not a usable proposition because I heavily use Numpy (and Scipy et al). So I am forced to use slow Python + fast Numpy or slow Numpy + fast Python. Very saddening. The C-Extension is just so off the pace, NumPyPy was meant to solve that quandry. And I know some smart Alec will trot out the usual 'downshift…
in short - funding. If we can find someone who wants fast numpy AND fast python under the same hood, we can combine the approaches of cpyext and numpypy and make it fast. The project is just too big to do on spare time. I've been trying to find some funding for that for quite a while, but I haven't been able to find any sizable backer just yet. Cheers, Maciej Fijalkowski
Here is the dumbest question in the world: our application has a gui which is pyqt, can we use pypy? Aside from pyqt, its completely vanilla python.
Many, many, many libraries already work fine with PyPy, so give it a shot.
I have a question and then a general vent 1. Does anyone know the latest update on NumPyPy? PyPy for me is just not a usable proposition because I heavily use Numpy (and Scipy et al). So I am forced to use slow Python + fast Numpy or slow Numpy + fast Python. Very saddening. The C-Extension is just so off the pace, NumPyPy was meant to solve that quandry. And I know some smart Alec will trot out the usual 'downshift…
What you could do is much simpler --- :
Split your Python application in two parts:
1. Keep your functions that make heavy use of Numpy and Scipy under CPython; expose your algorithms/functions as a web service/REST service/etc running under CPython.
2. The rest of the application, which of course needs to call the functions in (1), can be written in PyPy and call the web service in (1). This is where you would put the general-purpose stuff like web, graphics, database access, and of course all symbolic manipulations that do not require Numpy/Scipy.
I think this is a simple, workable solution.
Earlier quoted context omitted.
Python 3.6.1 is a lot faster than 3.3.2 which was used back when the heap benchmark was done. In my system, 2.7.10 vs 3.6.1: C:\Test\PyBench>py -2 test.py Done in 1188.308127 C:\Test\PyBench>py -3 test.py Done in 1454.897614 Please bring up to date benchmarks to the discussion, and stop complaining about old problems. Note: adjusted I the workload to be 1000 less iterations, to get the results fast for this comment,…
Python, R and other languages are just slower. If that is an issue I again will agree you need to move to another language. Can I ask what you need the speed for? How long are your reports running? Because a lot of times the reports run in under a few minutes and people just don't sub-set their data to code on. People feel it is "BIG DATA" when it is just annoying data that takes less then a minute to spit out.
I think there'll be more traction expanding the libraries for Julia, Nim, or Kotlin, all of which are much faster than Python, and similar in expressiveness. It's probably easier to create an optimization/ML/linear algebra/RNG/whatever library in Julia, Nim, or Kotlin than trying to get good performance out of R or Python.
I completely understand PyPy and numpy and why Python and R became popular, because there was a need for expressivity in the numerics space and other languages weren't offering it. But if you've been following them for long enough, it's clear that both of them had problems under the hood. I think people just crossed the boundary of appropriate use at some point, because the language syntax is so appealing for these sorts of things.
Maybe I'm wrong about all this and Pypy will deliver but I'm not holding my breath any longer. No offense to the Pypy developers--I'm immensely impressed with their work, and they've produced much more than I ever thought--but I do see some sort of asymptote. I think it would take some serious corporate influx of effort like what happened with javascript, and even then I worry that compatibility issues would rear their head. My guess is the Python 2-3 split would become a Python 2-3-Pypy split--maybe that's fine though.
Earlier quoted context omitted.
in short - funding. If we can find someone who wants fast numpy AND fast python under the same hood, we can combine the approaches of cpyext and numpypy and make it fast. The project is just too big to do on spare time. I've been trying to find some funding for that for quite a while, but I haven't been able to find any sizable backer just yet. Cheers, Maciej Fijalkowski
Maciej, how much would that require, ballpark? I think this is something there would be massive support for. My company would support it
I would say, ballpark, $100k. Email me at fijal at baroquesoftware.com
EDIT: the actual figure depends precisely on what you want, which parts of the stack have to be exactly how fast etc.
I have a question and then a general vent 1. Does anyone know the latest update on NumPyPy? PyPy for me is just not a usable proposition because I heavily use Numpy (and Scipy et al). So I am forced to use slow Python + fast Numpy or slow Numpy + fast Python. Very saddening. The C-Extension is just so off the pace, NumPyPy was meant to solve that quandry. And I know some smart Alec will trot out the usual 'downshift…
https://github.com/Microsoft/Pyjion
The core of it is there and it's been accepted into the main trunk, but we haven't had the cycles to do quality codegen yet.