Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone using PyPy for real work?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 185 posts

Re: Ask HN: Is anyone using PyPy for real work?

#41

Post by sodimel »

jsmeaton wrote: post #28
ADcorpo wrote: post #9

This post is a funny coincidence as I tried today to speed-up a CI pipeline running ~10k tests with pytest by switching to pypy. I am still working on it but the main issue is psycopg support for now, as I had to install psycopg2cffi in my test environment, but it will probably prevent me from using pypy for running our test suite, because psycopg2cffi does not have the same features and versions as psycopg2. This me…

Second this - no psycopg2 support and to a lesser extent lxml is a nonstarter and makes it pretty difficult to experiment with on production code bases. I could see a lot of adoption from Django deployments otherwise.

Yeah we don't use pypy for those exact reasons on our small django projects.

Re: Ask HN: Is anyone using PyPy for real work?

#42

Post by mattip »

macNchz wrote: post #30

I put PyPy in production at a previous job, running a pretty high traffic Flask web app. It was quick and pretty straightforward to integrate, and sped up our request timings significantly. Wound up saving us money because server load went down to process the same volume of requests, so we were able to spin down some instances. Haven’t used it in a bit mostly because I’ve been working on projects that haven’t had the…

You're welcome.

> that rely on incompatible extensions.

Which ones? Is using conda an option, we have more luck getting binary packages into their build pipelines than getting projects to build wheels for PyPI

Re: Ask HN: Is anyone using PyPy for real work?

#44

Post by [deleted] »

danpalmer wrote: post #18
ggm wrote: post #12

I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…

Debian is its own worst enemy with things like this. It’s why we eventually moved off it at a previous job, because deploying Python server applications on it was dreadful. I’m sure it’s better if you’re deploying an appliance that you hand off and never touch again, but for evolving modern Python servers it’s not well suited.

[deleted]

Re: Ask HN: Is anyone using PyPy for real work?

#45

Post by mattip »

wg0 wrote: post #39

While the community is here, anyone has embedded pypy as scriptable language for some larger program? Like Inkscape or scripting as part of a rule engine. Or for that, CPython is more suitable?

It is much easier to embed CPython, PyPy can only be embedded via CFFI [0].

[0] https://cffi.readthedocs.io/en/latest/embedding.html

Re: Ask HN: Is anyone using PyPy for real work?

#46

Post by syllogism »

ggm wrote: post #12

I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…

If you have very large dicts, you might find this hash table I wrote for spaCy helpful: https://github.com/explosion/preshed . You need to key the data with 64-bit keys. We use this wrapper around murmurhash for it: https://github.com/explosion/murmurhash

There's no docs so obviously this might not be for you. But the software does work, and is efficient. It's been executed many many millions of times now.

Re: Ask HN: Is anyone using PyPy for real work?

#48

Post by mattip »

waysa wrote: post #11

I used PyPy with SymPy when I was helping out a mathematician-friend. SymPy is not exactly fast, a free performance boost was very welcome.

Interesting. I was under the impression PyPy did not do so well with SymPy because the dynamic code paths are difficult to JIT. What kind tasks waw a speed up?

Re: Ask HN: Is anyone using PyPy for real work?

#49

Post by smashed »

silon42 wrote: post #35
gjvc wrote: post #24

Earlier quoted context omitted.

Yes 1000x What is it with them which makes them feel entitled to have special "dist-packages" vs "site-packages" as is the default? This drives me nuts, when I have a bunch of native packages I want to bundle in our in-house python deployment. CentOS and Ubuntu are vanilla, and only Debian (mind-boggingly) deviates from the well-trodden path. I still haven't figured out how to beat this dragon. All suggestions welcom…

dist packages are a must for software written in Python that is part of the distribution itself.

You're not really answering why they are important?

Is it because .deb packages will install inside dist-packages and when you run pip install as root without a virtual env, it installs inside site-packages?

I don't really see how this helps though? Sure you won't get paths to clash between the two but you still have duplicate packages which is probably not what you want..

Re: Ask HN: Is anyone using PyPy for real work?

#50

Post by _han »

I didn't hear about PyPy before, but I think you're doing great work.

I would be interested in seeing benchmarks where PyPy is compared with more recent versions of CPython. https://www.pypy.org/ currently shows a comparison with CPython 3.7, but recent releases of CPython (3.11+) put a lot of effort into performance which is important to take into account.

Post reply on HN