Live data from Hacker News

Ask HN: Is anyone using PyPy for real work?

news.ycombinator.com

11–20 of 185 posts

Re: Ask HN: Is anyone using PyPy for real work?

#12
I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks.

Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimised core datastructs pretty well.

Uplift from normal python was trivial. Most dev time spent fixing pip3 for pypy in debian not knowing what apts to load, with a lot of "stop using pip" messaging.

Re: Ask HN: Is anyone using PyPy for real work?

#14

I’ve been aware of it for a long time. I don’t use it. Why would I use it, what’s the compelling benefit?

Errm, nothing too serious. It's way faster for CPU bound code, and allows micro-threads.

This two weird tricks tend to create wonders, tho.

Re: Ask HN: Is anyone using PyPy for real work?

#15

I’ve been aware of it for a long time. I don’t use it. Why would I use it, what’s the compelling benefit?

From the project's homepage:

> A fast, compliant alternative implementation of Python

Performance without compromising too much on compatibility seems to be the main benefit. There is a talk on the YouTube channel «Pycon Sweden» from 5 years ago where the host showed some impressive speed gains for his workload (parsing black box dumps from planes).

Re: Ask HN: Is anyone using PyPy for real work?

#16
I liked Psyco a lot, it was totally awesome and with very few bugs (CPython differences) but that was looong ago. PyPy looks and feels like a monstrosity, it builds longer than most software for once, which is off-putting. I would be more interested in a Python JIT which is more like LuaJIT to Lua.

Re: Ask HN: Is anyone using PyPy for real work?

#17
I can't remember exactly what the use case was but we used at my old work (Start up providing a Web CDN/WAF type service, think the kind of stuff CloudFlare does nowadays) in ~2013 for some sort of batch processing analytics/billing type job, using MRJob and AWS Elastic Map Reduce over a seriously large data set.

The performance of PyPy over CPython saved us loads and loads time and thus $$$s, from what I can recall.

Re: Ask HN: Is anyone using PyPy for real work?

#18
post #12

I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…

Debian is its own worst enemy with things like this. It’s why we eventually moved off it at a previous job, because deploying Python server applications on it was dreadful.

I’m sure it’s better if you’re deploying an appliance that you hand off and never touch again, but for evolving modern Python servers it’s not well suited.

Re: Ask HN: Is anyone using PyPy for real work?

#19
post #12

I'm using pypy to analyse 350m DNS events a day, through python cached dicts to avoid dns lookup stalls. I am getting 95% dict cache hit rate, and use threads with queue locks. Moving to pypy definitely speeded me up a bit. Not as much as I'd hoped, it's probably all about string index into dict and dict management. I may recode into a radix tree. Hard to work out in advance how different it would be: People optimise…

Debian is its own worst enemy with things like this. It’s why we eventually moved off it at a previous job, because deploying Python server applications on it was dreadful. I’m sure it’s better if you’re deploying an appliance that you hand off and never touch again, but for evolving modern Python servers it’s not well suited.

[deleted]

Re: Ask HN: Is anyone using PyPy for real work?

#20
post #17

I can't remember exactly what the use case was but we used at my old work (Start up providing a Web CDN/WAF type service, think the kind of stuff CloudFlare does nowadays) in ~2013 for some sort of batch processing analytics/billing type job, using MRJob and AWS Elastic Map Reduce over a seriously large data set. The performance of PyPy over CPython saved us loads and loads time and thus $$$s, from what I can recall.

Thanks, that is hopeful, although quite a while ago.
Post reply on HN