We have evaluated PyPy but actually found Pyston to be more performant in most of our use cases (even with extensive JIT warming). That project unfortunately seems unmaintained now, but I am hoping that the improvements will be upstreamed into CPython.
Ask HN: Is anyone using PyPy for real work?
111–120 of 185 posts
Re: Ask HN: Is anyone using PyPy for real work?
#112I'm maintaining an internal change-data-capture application that uses a python library to decode mysql binlog and store the change records as json in the data lake (like Debezium). For our most busiest databases a single Cpython process couldn't process the amount of incoming changes in real time (thousands of events per second). It's not something that can be easily parallelized, as the bulk of the work is happening in the binlog decoding library (https://github.com/julien-duponchelle/python-mysql-replicati...).
So we've made it configurable to run some instances with Pypy - which was able to work through the data in realtime, i.e. without generating a lag in the data stream. The downside of using pypy was increased memory usage (4-8x) - which isn't really a problem. An actually problem that I didn't really track down was that the test suite (running pytest) was taking 2-3 times longer with Pypy than with CPython.
A few months ago I upgraded the system to run with CPython 3.11 and the performance improvements of 10-20% that come with that version now actually allowed us to drop Pypy and only run CPython. Which is more convenient and makes the deployment and configuration less complex.
Re: Ask HN: Is anyone using PyPy for real work?
#113You should probably put "Ask HN:" in your title. Personally I don't use PyPy for anything, though I have followed it with interest. Most of the things I need to go faster are numerical, so Numba and Cython seem more appropriate.
Cut him some slack, he's only been registered for 10 years
I read this as humor and I imagine mattip may have done also.
Re: Ask HN: Is anyone using PyPy for real work?
#114I don't. I work for a company where we always try to track the latest stable version of Python. Right now we are on 3.11, and unfortunately Pypy is lagging behind.
Re: Ask HN: Is anyone using PyPy for real work?
#115At Alooma (https://www.linkedin.com/mwlite/company/alooma) we've been running all our integrations with data sources using PyPy.
Main motivation was indeed performance gains.
FWIW, since I've seen it mentioned, we've also been using psycopg2cffi to access Postgres sources.
The product now lives (at least partially) as Datastream on GCP (https://cloud.google.com/datastream/docs/overview). I'm not sure though if it's still running on PyPy.
I could try and connect with the folks still working on it, if you're interested.
Re: Ask HN: Is anyone using PyPy for real work?
#116Earlier quoted context omitted.
For a more apples to apples comparison, you would install pypy using your package manager, e.g. apt install pypy3 or brew install pypy3 . On Linux, you might have to add a package repo first.
I find that much scarier to do personally since it seems a lot more likely to screw up other stuff on your machine, whereas with pyenv it's all self-contained in the venv. Also using apt packages tends to install a pretty old version.
No, installing a package with apt is not more likely to screw up your machine than installing it manually. Moreover, you seem to be completely fine using the apt-installed CPython, while you think PyPy needs to be installed manually.
Re: Ask HN: Is anyone using PyPy for real work?
#117At Alooma ( https://www.linkedin.com/mwlite/company/alooma ) we've been running all our integrations with data sources using PyPy. Main motivation was indeed performance gains. FWIW, since I've seen it mentioned, we've also been using psycopg2cffi to access Postgres sources. The product now lives (at least partially) as Datastream on GCP ( https://cloud.google.com/datastream/docs/overview ). I'm not sure though if it…
Cool. Yes, I am interested in hearing more.
Re: Ask HN: Is anyone using PyPy for real work?
#118We use PyPy for performing verification of our software stack [1], and also for profiling tools [2]. The verification tool is basically a complete reimplementation of our main product, and therefore encodes a massive amount of business logic (and therefore difficult to impossible to rewrite in another language). As with other users, we found the switch to PyPy was seamless and provides us with something like a 2.5x speedup out of the box, with (I think) higher speedups in some specific cases.
We eventually rewrote the profiler tool in Rust for additional speedups, but as mentioned for the verification engine, it's probably too complicated to ever do that so we really appreciate drop-in tools like PyPy that can speed up our code.
[1]: https://github.com/StanfordLegion/legion/blob/master/tools/l...
[2]: https://github.com/StanfordLegion/legion/blob/master/tools/l...
Re: Ask HN: Is anyone using PyPy for real work?
#119A bit meta. It seems like it would be nice to have no-action tickets for open source projects. Quite often you would want to just thank somebody, or say that you would prefer it that way and don't understand why is it this way or it would be cool to have this or that, but of course opening ticket on github feels like wasting time of the maintainer and especially when you have some feedback like e.g. what would you li…
You can comment on our blog, or open an issue. Frankly, we get so little feedback that dealing with new issues is not a hassle.
Re: Ask HN: Is anyone using PyPy for real work?
#120I used PyPy extensively at a previous employer. The use case was to accelerate an application that was CPU-bound because of serde, which could not be offloaded using multiprocessing. PyPy resulted in a 10x increase in message throughput, and made the project viable in python. Without PyPy, we would have rebuilt the application in Java.