I wonder if programs like Rye, that distribute python in a way similar to Rust's rustup, can help. Rye already supports pypy, you can just pull down pypy3.9 at will into any particular python project managed by rye.
Ask HN: Is anyone using PyPy for real work?
21–30 of 185 posts
Re: Ask HN: Is anyone using PyPy for real work?
#22I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…
> it's probably all about string index into dict and dict management
Cool. Is the performance here something you would like to pursue? If so could you open an issue [0] with some kind of reproducer?
Re: Ask HN: Is anyone using PyPy for real work?
#23I use CPython most of the time but PyPy was a real lifesaver when I was doing a project that bridged EMOF and RDF, particularly I was working with moderately sized RDF models (say 10 million triples) with rdflib. With CPython, I was frustrated with how slow it was, and complained about it to the people I was working with, PyPy was a simple upgrade that sped up my code to the point where it was comfortable to work wit…
What do you use RDF models for?
Re: Ask HN: Is anyone using PyPy for real work?
#24I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…
Debian is its own worst enemy with things like this. It’s why we eventually moved off it at a previous job, because deploying Python server applications on it was dreadful. I’m sure it’s better if you’re deploying an appliance that you hand off and never touch again, but for evolving modern Python servers it’s not well suited.
Yes 1000x What is it with them which makes them feel entitled to have special "dist-packages" vs "site-packages" as is the default? This drives me nuts, when I have a bunch of native packages I want to bundle in our in-house python deployment. CentOS and Ubuntu are vanilla, and only Debian (mind-boggingly) deviates from the well-trodden path.
I still haven't figured out how to beat this dragon. All suggestions welcome!
Re: Ask HN: Is anyone using PyPy for real work?
#25I use it at work for a script that parses and analyzes some log files in an unusual format. Wrote a naive parser with a parsing combinator library. It was too slow to be usable with CPython. Tried PyPy and got a 50x speed increase (yes, 50 times faster). Very happy with the results, actually =)
Re: Ask HN: Is anyone using PyPy for real work?
#26We don't. To be honest, I didn't realize PyPy supported Python 3. I thought it was eternally stuck on Python 2.7.
So the good: It apparently now supports Python 3.9? Might want to update your front page, it only mentions Python 3.7.
The bad: It only supports Python 3.9, we use newer features throughout our code, so it'd be painful to even try it out.
Re: Ask HN: Is anyone using PyPy for real work?
#27I've never used it because the (unknown) effort of switching and the chance of compatibility issues have always made it unappealing compared to just switching to a faster language.
If I could just `pip3 install pypy` and then set an environment variable to use it or something like that then I'd give it a try. It does feel a bit like adding a jet pack to a rowing boat though. I know some people use Python in situations where the performance requirement isn't "I literally don't care" but surely not very many?
Obviously if it was the default that would be fantastic.
Re: Ask HN: Is anyone using PyPy for real work?
#28This post is a funny coincidence as I tried today to speed-up a CI pipeline running ~10k tests with pytest by switching to pypy. I am still working on it but the main issue is psycopg support for now, as I had to install psycopg2cffi in my test environment, but it will probably prevent me from using pypy for running our test suite, because psycopg2cffi does not have the same features and versions as psycopg2. This me…
Second this - no psycopg2 support and to a lesser extent lxml is a nonstarter and makes it pretty difficult to experiment with on production code bases. I could see a lot of adoption from Django deployments otherwise.
Re: Ask HN: Is anyone using PyPy for real work?
#29I use it for data transformation, cleanup and enrichment. (TXT, CSV, Json, XML, database) to (TXT, CSV, JSON, XML, database).
Speed up of 30x - 40x. The highest speedup on those that require logic in the transformation. (lot of function calls, numerical operations and dictionary lookups).
Re: Ask HN: Is anyone using PyPy for real work?
#30I put PyPy in production at a previous job, running a pretty high traffic Flask web app. It was quick and pretty straightforward to integrate, and sped up our request timings significantly. Wound up saving us money because server load went down to process the same volume of requests, so we were able to spin down some instances.
Haven’t used it in a bit mostly because I’ve been working on projects that haven’t had the same bottleneck, or that rely on incompatible extensions.
Thank you for your work on the project!