Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyPy 2.0 Released

morepypy.blogspot.com

Post reply on HN
Print view View original

51–60 of 77 posts

Re: PyPy 2.0 Released

#51

Post by buster »

cpeterso wrote: post #38
darkarmani wrote: post #37

Earlier quoted context omitted.

Because who cares of existing, running, legacy code, right?

Do you mean the people who haven't upgraded from Python 2.7 to Python 3?

The people who haven't upgraded to Python 3 in their projects are probably mostly people with dependencies on py2.7 libraries. Happened to me in my last project. I started using pypy until some bug in pypy prevented me from using an important package. Switched to python 3 until some other dependency wasn't available. So it's a pity, but i was more or less forced to use py2.7 (without putting major effort into 3rd party libraries).

Re: PyPy 2.0 Released

#52

Post by mattip »

CJefferson wrote: post #50
adamt wrote: post #46

Whilst I applaud the efforts of PyPy, I've always been disappointed with the performance benefit it's given me with real world code. Specifically: * PyPy Dict performance is very slow (slower than Cpython). A lot of the code I need to write that is cpu-intensive python code processes data (from stuff like logs) into various data structures that use dicts. Pypy is rarely more than 10% faster and sometimes slower. * Th…

I've had exactly the same problem. I have had tree-manipluation AI problems which run for > 30 minutes (seems like an ideal thing for pypy, before I rewrite them in C++). pypy is almost always slower, and never more than about 15% faster, whereas a simple line-by-line C++ rewrite can be 20x faster.

Please help PyPy by providing an example benchmark. The project is test and data driven, and the more data it has the better it can be

Re: PyPy 2.0 Released

#54

Post by apendleton »

adamt wrote: post #46

Whilst I applaud the efforts of PyPy, I've always been disappointed with the performance benefit it's given me with real world code. Specifically: * PyPy Dict performance is very slow (slower than Cpython). A lot of the code I need to write that is cpu-intensive python code processes data (from stuff like logs) into various data structures that use dicts. Pypy is rarely more than 10% faster and sometimes slower. * Th…

Have you used pypy recently? I've found that memory usage in particular is much better as of around 1.9, compared to previous releases. Still worse than CPython, for sure, but some of my code is around 10x faster under pypy (all depends on what I'm doing, though, for sure; this stuff is numerically heavy).

Re: PyPy 2.0 Released

#55

Post by nickik »

rcthompson wrote: post #27
MichailP wrote: post #8

Earlier quoted context omitted.

For example does PyPy always warm up, or can it store "warmed up" version of some function (where type of objects is inferred)? I know that this is not in accordance with highly dynamic nature of Python, but not all functions are highly dynamic.

My understanding (from a previous HN discussion about javascript) is that since the output from a "just-in-time compiler" is actually machine code generated on the fly, it includes direct references to memory locations that are only valid for the lifetime of the process. So the output of a JIT is simply not in a format that can be saved and reloaded later.

Actually its possible to have a JIT like that but it is uncommon.

Re: PyPy 2.0 Released

#56

Post by MichailP »

Derbasti wrote: post #39
MichailP wrote: post #3

Can someone knowledgeable compare PyPy, Numba and Cython? I mostly use Cython. Tried Numba also, it has very nice workflow when it works with autojit (when it doesn't error messages are pretty cryptic). With PyPy I don't understand why for example all that type information obtained from jit wouldn't be used to make something like Numba specialized functions or Cython modules (noob question but please answer).

Have you tried the cffi? I used to use cython, but since I found the cffi, I haven't looked back.

No, I got scared away by Simple example and Real example from the docs :) Do you maybe have some hints/links for cffi?

Re: PyPy 2.0 Released

#57

Post by kenko »

illumen wrote: post #49
adamt wrote: post #46

Whilst I applaud the efforts of PyPy, I've always been disappointed with the performance benefit it's given me with real world code. Specifically: * PyPy Dict performance is very slow (slower than Cpython). A lot of the code I need to write that is cpu-intensive python code processes data (from stuff like logs) into various data structures that use dicts. Pypy is rarely more than 10% faster and sometimes slower. * Th…

The benchmarks are a little bit misleading for some types of code. They do not measure single run code well. In these cases pypy doesn't do as well as when the JIT has warmed up. The interpreter is slower than the CPython one. For code which doesn't JIT well, it will go slower with pypy. The firefox JS engine now has a fast interpreter, a quick JIT, and a more optimizing one. This means the baseline performance is be…

"The firefox JS engine now has a fast interpreter, a quick JIT, and a more optimizing one. This means the baseline performance is better in CPython"

Huh?

Re: PyPy 2.0 Released

#58

Post by andreasvc »

DasIch wrote: post #5
MichailP wrote: post #3

Can someone knowledgeable compare PyPy, Numba and Cython? I mostly use Cython. Tried Numba also, it has very nice workflow when it works with autojit (when it doesn't error messages are pretty cryptic). With PyPy I don't understand why for example all that type information obtained from jit wouldn't be used to make something like Numba specialized functions or Cython modules (noob question but please answer).

Cython is a compiler not an interpreter. Numba requires explicit hinting and can only optimize some undefined (?) subset of Python. PyPy on the other hand is simply a fast Python interpreter. Not sure what you mean regarding the type information gathered by the JIT.

Cython is not actually a compiler. It is essentially a more convenient way of writing C/C++ extension modules. It produces C/C++ glue code which is then compiled by an actual compiler like gcc. By tapping into the existing C API of cpython it is simpler than PyPy & NumPy which work on a lower level, but on the other hand Cython does little optimization for you.

Re: PyPy 2.0 Released

#59

Post by ldng »

CJefferson wrote: post #50
adamt wrote: post #46

Whilst I applaud the efforts of PyPy, I've always been disappointed with the performance benefit it's given me with real world code. Specifically: * PyPy Dict performance is very slow (slower than Cpython). A lot of the code I need to write that is cpu-intensive python code processes data (from stuff like logs) into various data structures that use dicts. Pypy is rarely more than 10% faster and sometimes slower. * Th…

I've had exactly the same problem. I have had tree-manipluation AI problems which run for > 30 minutes (seems like an ideal thing for pypy, before I rewrite them in C++). pypy is almost always slower, and never more than about 15% faster, whereas a simple line-by-line C++ rewrite can be 20x faster.

Out of curiosity, was your python code relying on dicts or using structured classes ? It looks like Pypy is better at optimizing classes than dictionaries. Which might explain to some degree why people are not seeing the perf improvement they expect. A lot of existing python code rely on dict manipulation which gives decent perf on CPython where classes would play nicer with Pypy.

Re: PyPy 2.0 Released

#60

Post by pjmlp »

keeperofdakeys wrote: post #36
pjmlp wrote: post #4

Congratulations! Looking forward to the day PyPy becomes the reference implementation.

In many cases, you want to be careful about using JIT'd platforms. Many people forget that the memory requirements skyrocket. When you want to use a language on many different platforms, including embedded, you start to see how having a JIT interpreter as your reference platform can be disadvantageous. CPython isn't exactly slow either, especially when you consider many 'intensive' modules are written directly in C.…

You are right, but having a JIT able to generate fast code means you can avoid writing C code at all, just look how fast Julia already is.
Post reply on HN