Ask HN: Is anyone using PyPy for real work?

We use PyPy extensively at my employer, a small online retailer, for the website, internal web apps, ETL processes, and REST API integrations.

We use the PyPy provided downloads (Linux x86 64 bit) because it's easier to maintain multiple versions simultaneously on Ubuntu servers. The PyPy PPA does not allow this. I try to keep the various projects using the latest stable version of PyPy as they receive maintenance, and we're currently transitioning from 3.9/v7.3.10 to 3.10/v7.3.12.

Thank you for all of the hard work providing a JITed Python!

Lots of questions :)

For c-extensions see https://www.pypy.org/posts/2018/09/inside-cpyext-why-emulati...

We would like to be able to "just JIT" better. But for that we need feedback about what is still unreasonably slow, and resources to work on improving it. Right now PyPy is on a shoe-string budget of volunteers.

For rust, like CPython, use PyO3, which works with PyPy.

I am not sure about Pydantic. Sounds like a topic for someone to investigate on their codebase and tell us how PyPy does.

Cool. Would love to hear more about the successes and problems, or even get a guest blog post on https://www.pypy.org/blog/

Basically afaik the default C API (`Python.h`) is matched to CPython's internal representations, hence it's a pain to support it for alternative implementations and incurs cost penalties. The preferred way to interact with C code in pypy is through cffi (https://cffi.readthedocs.io/en/latest/) and ctypes (which afaik is implemented in pure python on top of cffi in pypy).

Numpy being itself written in C and C++ it is strongly tied to the C API and has a complicated build process. Some stuff works and some don't (didn't try recently). If you're invested in numerical python you should most likely not use pypy but go for stuff like cython (like scipy does).

For psycopg apparently you can use psycopg2cffi (never tried).

> How does pypy handle packages written in other languages, like rust? can I use pypy if I depend on Pydantic?

PyO3 supports pypy so everything should be fine.

So I wrote this library

https://paulhoule.github.io/gastrodon/

which makes it very easy to visualize RDF data with Jupyter by turning SPARQL results into data frames.

Here are two essays I wrote using it

https://ontology2.com/essays/LookingForMetadataInAllTheWrong...

https://ontology2.com/essays/PropertiesColorsAndThumbnails.h...

People often think RDF never caught on but actually there are many standards that are RDF-based such as RSS, XMP, ActivityPub and such that you can work on quite directly with RDF tools.

Beyond that I’ve been on a standards committee for ISO 20022 where we’ve figured out, after quite a few years of looking at the problem, how to use RDF and OWL as a master standard for representing messages and schemas in financial messaging. In the project that needed PyPy we were converting a standard represented in EMOF into RDF. Towards the end of last year I figured out the right way to logically model the parts of those messages and the associated schema with OWL. That is on its way of becoming one of those ISO standard documents that unfortunately costs 133 swiss franc. I also figured out that it is possible to do the same for many messages defined with XSLT and I’m expecting to get some work applying this to a major financial standard and I think there will be some source code and a public report on that.

Notably the techniques I use address quite a few problems with the way most people use RDF, most notably many RDF users don’t use the tools available to represented ordered collections, a notable example with this makes trouble is in Dublin Core for document (say book) metadata where you can’t represent the order of the authors of a paper which is something the authors usually care about a great deal. XMP adapts the Dublin Core standard enough to solve this problem, but with the techniques I use you can use RDF to do anything any document database can, though some SPARQL extensions would make it easier.