Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

morepypy.blogspot.com

Post reply on HN
Print view View original

31–40 of 150 posts

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#31

Post by coldtea »

MatthewWilkes wrote: post #9

> We would also like to thank our contributors (7 new ones since PyPy 2.6.0) and encourage new people to join the project. PyPy has many layers and we need help with all of them This doesn't match my experience of the PyPy project. I found a tiny bug in the stdlib matching against CPython, went into IRC to ask a question about test running to be sure I got it right and was quickly engaged in conversation about why I…

>This doesn't match my experience of the PyPy project

One experience, with one developer, and without going through official channels to asks for participation etc.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#32

Post by sandGorgon »

fijal wrote: post #6
wyldfire wrote: post #5

Kudos to the pypy team! I suspect pypy would benefit from some kind of marketing campaign. It rarely lets me down and yet lots of python devs don't know about it. The most popular package(s) that don't work with it have been numpy, scipy, etc. When I first encountered pypy, there was a numpypy and I wouldn't be surprised if that's different now. But bottom line, tons of python code out there needs more performance an…

we're trying to address both numpy and C extensions, stay tuned ;-)

you should kickstarter this - and also the python 3 support.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#33

Post by dorfsmay »

forgotpwtomain wrote: post #25
MrBuddyCasino wrote: post #24

Earlier quoted context omitted.

He didn't want to report a bug, he wanted to contribute to the project. I can think of a few reasons why. They lost a new potential committer by "being pragmatic" about it, but ignoring inter-personal relationships, like so often.

Personally I'd be quite excited to see a bug I reported fixed the same-day - of course people's expectations are different it's not always easy to balance these things.

One of the rewards of participating in OSS is to make your name visible, this can then be used as creds to gain commit rights or to talk at conferences as well as proof of competence when looking for a job.

Interestingly, I've had the same experience when reporting and asking how to contribute to Python 3, not by a core dev but somebody "stealing" my bug and quickly submitting a patch!

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#34

Post by fijal »

canjobear wrote: post #28

It's hard to tell from these posts under what circumstances will SIMD vectorization kick in outside of numpy. E.g., will it work for plain old list comprehensions, which are everywhere in my code? That would be awesome.

Generally speaking "no", but there are a few cases where it might, e.g. if you're using arrays/numpy arrays in list comprehensions. We could not get it to run on normal lists for a bit obscure reasons.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#35

Post by dorfsmay »

fijal wrote: post #27
gdulli wrote: post #26

Earlier quoted context omitted.

I've only used simplejson. A mapreduce job could be reading billions of lines and possibly doing very simple logic on each line, so the parsing/encoding is a big percentage of the processing, sometimes nearly all of it. I'll try 4.0 and see how it does now.

Try builtin json instead of simplejson for pypy (it's API-equivalent)

On big jobs (millions of json input/output) I have found ujson significantly faster than json + pypy.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#36

Post by DasIch »

coldtea wrote: post #31
MatthewWilkes wrote: post #9

> We would also like to thank our contributors (7 new ones since PyPy 2.6.0) and encourage new people to join the project. PyPy has many layers and we need help with all of them This doesn't match my experience of the PyPy project. I found a tiny bug in the stdlib matching against CPython, went into IRC to ask a question about test running to be sure I got it right and was quickly engaged in conversation about why I…

> This doesn't match my experience of the PyPy project One experience, with one developer, and without going through official channels to asks for participation etc.

The IRC channel is an official channel, in fact it's by far the most important one of the PyPy project.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#37

Post by MatthewWilkes »

forgotpwtomain wrote: post #25
MrBuddyCasino wrote: post #24

Earlier quoted context omitted.

He didn't want to report a bug, he wanted to contribute to the project. I can think of a few reasons why. They lost a new potential committer by "being pragmatic" about it, but ignoring inter-personal relationships, like so often.

Personally I'd be quite excited to see a bug I reported fixed the same-day - of course people's expectations are different it's not always easy to balance these things.

I can understand that, I certainly would have been stoked about a bug report being fixed on the same day; I hadn't reported the bug though. I was working on providing a bug report with a patch alongside it and (perhaps foolishly) mentioned the bug informally as context as to why I was doing this.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#38

Post by DasIch »

omginternets wrote: post #10

Guys, what's a "trace optimizer"? Google is failing me by returning results about MySQL.

Traces are an artifact produced by some JITs.

Basically you observe loops and loops that satisfy certain criteria are traced, that is all operations performed within one iteration are stored. Such a trace is then optimized, compiled to machine code and then executed for every iteration when the loop is encountered again.

Re: PyPy 4.0.0 Released – A Jit with SIMD Vectorization and More

#39

Post by pjmlp »

wyldfire wrote: post #29
dr_zoidberg wrote: post #17

Earlier quoted context omitted.

I think you guys are saying the same thing. Ie, because they don't need to switch languages, and PyPy is a JIT engine for Python, these tons of Python code would benefit from if the devs were aware of PyPy .

Indeed that's what I meant and perhaps pjmlp thinks that the feature set in numpy could have been / could be implemented in pure python, now that we have PyPy.

What I mean is that I see PyPy very positive, as a way to stay in Python instead of jumping into C, Go or whatever language.

In terms of performance naysayers only get convinced when someone proves them wrong, not because of what we think might be possible.

So it is important that we have research in GC systems, JIT compilers for dynamic languages, optimizers for FP and LP languages and so on.

Otherwise we might as well keep on using just Assembly.

Post reply on HN