Maciej Fijalkowski's view on PyPy's future
lostinjit.blogspot.com
Maciej Fijalkowski's view on PyPy's future
1–10 of 22 posts
Re: Maciej Fijalkowski's view on PyPy's future
#2PyPy will likely be complete enough to use by the end of 2012, if not sooner. By that I mean most of the major libraries will work with it and people will ask themselves "why wouldn't I want a massive speedup?". The one reason why they might not is memory usage: it can be 10 times as much. That's significant in quite a few applications.
As it's beyond my ken I would like to know from someone capable of answering if this is going to be a constant constraint, stemming from the design of the project, or if memory usage is likely to go down at some point.
Re: Maciej Fijalkowski's view on PyPy's future
#3What should we make of the failure of PyPy's users to fund Maciej's work? The value of PyPy seems much, much greater than one engineer's salary.
Re: Maciej Fijalkowski's view on PyPy's future
#4PyPy will likely be complete enough to use by the end of 2012, if not sooner. By that I mean most of the major libraries will work with it and people will ask themselves "why wouldn't I want a massive speedup?". The one reason why they might not is memory usage: it can be 10 times as much. That's significant in quite a few applications. As it's beyond my ken I would like to know from someone capable of answering if t…
I believe the memory usage has been adressed lately (either last release, or shortly after). There was an article about it on HN, which i can't find atm. Do you mean some specific use cases?
Re: Maciej Fijalkowski's view on PyPy's future
#5What should we make of the failure of PyPy's users to fund Maciej's work? The value of PyPy seems much, much greater than one engineer's salary.
I find it puzzling as well. Google heavily uses python, as do a number of other web companies. Further, Ubuntu and Redhat have it as a system administration language. It's almost the default language for O'Reilly books that aren't language specific. Given all that you'd think there'd be a few more corporate contributions now that a x5 speedup has already been proven and it seems to be just a matter of polishing it up.
A bit of searching and it seems the memory problems I raised in my other comment aren't so drastic after all. Theoretically it can use less memory in many operations and the current blowouts are not as high as I thought (I was going from one benchmark a HNer, brianh, was kind enough to run for me[1]).
Re: Maciej Fijalkowski's view on PyPy's future
#6PyPy will likely be complete enough to use by the end of 2012, if not sooner. By that I mean most of the major libraries will work with it and people will ask themselves "why wouldn't I want a massive speedup?". The one reason why they might not is memory usage: it can be 10 times as much. That's significant in quite a few applications. As it's beyond my ken I would like to know from someone capable of answering if t…
I believe the memory usage has been adressed lately (either last release, or shortly after). There was an article about it on HN, which i can't find atm. Do you mean some specific use cases?
http://news.ycombinator.com/item?id=3349429
If you search for my username in that thread I asked for someone to run the code from the story on a similar dataset with PyPy. Twice as fast but x10 memory usage. But as I said elsewhere a bit of searching suggests it won't be a showstopper.
Re: Maciej Fijalkowski's view on PyPy's future
#7In my opinion, they could get more contributors if they simplified the rather byzantine build process for pypy. It's especially annoying for developers attempting to port to other platforms.
(No incremental build, it takes roughly three hours on a high-end nehalem workstation, and if it fails for any reason, you get to start all over again!)
Re: Maciej Fijalkowski's view on PyPy's future
#8What should we make of the failure of PyPy's users to fund Maciej's work? The value of PyPy seems much, much greater than one engineer's salary.
I find it puzzling as well. Google heavily uses python, as do a number of other web companies. Further, Ubuntu and Redhat have it as a system administration language. It's almost the default language for O'Reilly books that aren't language specific. Given all that you'd think there'd be a few more corporate contributions now that a x5 speedup has already been proven and it seems to be just a matter of polishing it up…
> I find it puzzling as well. Google heavily uses python, as do a number of other web companies.
Especially after Google invested engineers on Unladen Swallow, which fizzled out. I've read that Google likes to max out their servers to the point where OOM is not unlikely, so PyPy memory usage might not be worth the runtime performance gains.
Re: Maciej Fijalkowski's view on PyPy's future
#9Earlier quoted context omitted.
I believe the memory usage has been adressed lately (either last release, or shortly after). There was an article about it on HN, which i can't find atm. Do you mean some specific use cases?
http://news.ycombinator.com/item?id=3349429 If you search for my username in that thread I asked for someone to run the code from the story on a similar dataset with PyPy. Twice as fast but x10 memory usage. But as I said elsewhere a bit of searching suggests it won't be a showstopper.
Be careful extrapolating from such small examples, where the memory usage is dominated by fixed overhead. Running Django or something similar is more likely to give a real world result, and even then, a bunch of tests is far better.
3mb vs 30mb might be par for the course for smaller programs, but either is negligible.
Re: Maciej Fijalkowski's view on PyPy's future
#10PyPy will likely be complete enough to use by the end of 2012, if not sooner. By that I mean most of the major libraries will work with it and people will ask themselves "why wouldn't I want a massive speedup?". The one reason why they might not is memory usage: it can be 10 times as much. That's significant in quite a few applications. As it's beyond my ken I would like to know from someone capable of answering if t…
PyPy has higher memory usage than CPython for two main reasons: A) JIT-generated code, B) pure garbage collection lets dead objects live longer than CPython's refcount/GC hybrid. Neither of these is going to go away, although it's reasonable to expect the gap to narrow over time. More importantly, much of this extra memory is a fixed cost, so while simple programs will see a 10x increase, real-world programs should see a less dramatic impact.
FWIW, my own tests of pypy 1.7 have shown the memory overhead to be about 3x: http://groups.google.com/group/python-tornado/browse_thread/...