Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is anyone using PyPy for real work?

news.ycombinator.com

Post reply on HN
Print view View original

181–185 of 185 posts

Re: Ask HN: Is anyone using PyPy for real work?

#181

Post by syllogism »

ggm wrote: post #158
syllogism wrote: post #46

Earlier quoted context omitted.

If you have very large dicts, you might find this hash table I wrote for spaCy helpful: https://github.com/explosion/preshed . You need to key the data with 64-bit keys. We use this wrapper around murmurhash for it: https://github.com/explosion/murmurhash There's no docs so obviously this might not be for you. But the software does work, and is efficient. It's been executed many many millions of times now.

I'm in strings, not 64 bit keys. But thanks, nice to share ideas.

The idea is to hash the string into a 64-bit key. You can store the string in a value, or you can have a separate vector and make the value a struct that has the key and the value.

The chance of colliding on the 64-bit space is low if the hash distributes evenly, so you just yolo it.

Re: Ask HN: Is anyone using PyPy for real work?

#182

Post by saila »

eigenvalue wrote: post #110
saila wrote: post #103

Earlier quoted context omitted.

For a more apples to apples comparison, you would install pypy using your package manager, e.g. apt install pypy3 or brew install pypy3 . On Linux, you might have to add a package repo first.

I find that much scarier to do personally since it seems a lot more likely to screw up other stuff on your machine, whereas with pyenv it's all self-contained in the venv. Also using apt packages tends to install a pretty old version.

I use pyenv myself, but that is beside the point. The two examples above are using different strategies to install python3 versus pypy. A valid comparison would use a package manager for both or pyenv for both.

Re: Ask HN: Is anyone using PyPy for real work?

#183

Post by waysa »

mattip wrote: post #48
waysa wrote: post #11

I used PyPy with SymPy when I was helping out a mathematician-friend. SymPy is not exactly fast, a free performance boost was very welcome.

Interesting. I was under the impression PyPy did not do so well with SymPy because the dynamic code paths are difficult to JIT. What kind tasks waw a speed up?

It's been a while and the code is long lost. We only touched the surface of SymPy. Functions, Substitutions, some `ingegrate` and `simplify` is what I remember. The maths was already done. My job was to verify some equations.

Re: Ask HN: Is anyone using PyPy for real work?

#184

Post by fragebogen »

mattip wrote: post #166
fragebogen wrote: post #164

I'm running a constrained convex optimization project at work, where we need as close to real time ( Basically I'm using a SciPy exclusively for the optimization routine: * minimize(method="SLSQP") [0] * A list comprehention which calls ~10-500 pre-fitted PchipInterpolator [1] functions and stores the values as a np.array(). The Pchip functions (and it's first derivatives) are used in the main opt function as well as…

If you find your bottlenecks in SciPy or Numpy, then PyPy will not help. Those are primarily written in C, so the PyPy JIT cannot peer inside and do any magic.

My effort was rather in trying to speed up all the python looping, etc. around the np calls. But I never went far trying to actually benchmark the entire pipeline in order to find out what was the actual bottleneck.
Post reply on HN