61–70 of 185 posts
I use it at work for a script that parses and analyzes some log files in an unusual format. Wrote a naive parser with a parsing combinator library. It was too slow to be usable with CPython. Tried PyPy and got a 50x speed increase (yes, 50 times faster). Very happy with the results, actually =)
Thanks for the feedback. It does seem like parsing logs and simulations is a sweet spot for PyPy
I've deployed used the pypy:3.9 image on docker.
One thing I did notice is that it was significantly faster on my local machine vs when I tried to deploy it using an AWS lambda/fargate. I know this is because of virtualization/virtual-cpu, but there was not much I could do to improve it.
Also in my day job we use pypy in all our python deployments, to be fair until now I thought that everybody would develop in python, test in pypy for an easy speed boost and only got back to python if pypy was slower than cpython
Earlier quoted context omitted.
dist packages are a must for software written in Python that is part of the distribution itself.
You're not really answering why they are important? Is it because .deb packages will install inside dist-packages and when you run pip install as root without a virtual env, it installs inside site-packages? I don't really see how this helps though? Sure you won't get paths to clash between the two but you still have duplicate packages which is probably not what you want..
Debian also tries to ship just one version of everything in a single distribution release to reduce the burden on its maintainers.
This is fundamentally at odds with pip. If you've pip installed something, then that'll likely be the latest version of that package, and in the general case won't be the version of the same thing that shipped in the Debian release. If there exist debs that depend on that package and they are shared between pip and debs, now the deb could be using a different version of the dependency than the deb metadata says is acceptable, leading to breakage.
Another way of putting this: it shouldn't be possible for you to pip upgrade a dependency that a deb shipped by Debian itself relies upon. Because then you'd creating a Frankenstein system where Debian cannot rely on its own dependencies providing what it expects.
This is fixed by having two places where things are installed. One for what the system package manager ships, and one for your own use with pip and whatever you want to do. In this sense, having duplicate packages is actually exactly what you want.
Earlier quoted context omitted.
dist packages are a must for software written in Python that is part of the distribution itself.
You're not really answering why they are important? Is it because .deb packages will install inside dist-packages and when you run pip install as root without a virtual env, it installs inside site-packages? I don't really see how this helps though? Sure you won't get paths to clash between the two but you still have duplicate packages which is probably not what you want..
If you pip install requests globally, you just broke a few of your distrib's packages.
I put PyPy in production at a previous job, running a pretty high traffic Flask web app. It was quick and pretty straightforward to integrate, and sped up our request timings significantly. Wound up saving us money because server load went down to process the same volume of requests, so we were able to spin down some instances. Haven’t used it in a bit mostly because I’ve been working on projects that haven’t had the…
You're welcome. > that rely on incompatible extensions. Which ones? Is using conda an option, we have more luck getting binary packages into their build pipelines than getting projects to build wheels for PyPI
If we could use pypy, while still using those packages, I think it'd be the go-to interpreter. Why can't pypy optimize everything else, and leave the C stuff as-is?
How does pypy handle packages written in other languages, like rust? can I use pypy if I depend on Pydantic?
I've never used it because the (unknown) effort of switching and the chance of compatibility issues have always made it unappealing compared to just switching to a faster language. If I could just `pip3 install pypy` and then set an environment variable to use it or something like that then I'd give it a try. It does feel a bit like adding a jet pack to a rowing boat though. I know some people use Python in situation…
rtx use python@pypy3.10
I use it at work for a script that parses and analyzes some log files in an unusual format. Wrote a naive parser with a parsing combinator library. It was too slow to be usable with CPython. Tried PyPy and got a 50x speed increase (yes, 50 times faster). Very happy with the results, actually =)
This post is a funny coincidence as I tried today to speed-up a CI pipeline running ~10k tests with pytest by switching to pypy. I am still working on it but the main issue is psycopg support for now, as I had to install psycopg2cffi in my test environment, but it will probably prevent me from using pypy for running our test suite, because psycopg2cffi does not have the same features and versions as psycopg2. This me…