Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

71–80 of 211 posts

Re: Pyston v2: Faster Python

#71

Post by sk2020 »

throwaway894345 wrote: post #51
sk2020 wrote: post #44

Earlier quoted context omitted.

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention). WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The ref…

So who does it right? If all these APIs suck compared to an imaginary perfect library, then that isn’t a useful comparison.

Also, if an endpoint is spending minutes to respond, then I would think actually profiling the application would be a good start. Maybe researching prior art in the problem domain would be good too. If nobody can be bothered to explore the several solutions to distributing pandas computations over multiple cores, like Dask, and get the NPV of just buying more or faster cores, then “Python sucks” isn’t your problem.

Re: Pyston v2: Faster Python

#72

Post by newen »

hhas01 wrote: post #45
qorrect wrote: post #6

Earlier quoted context omitted.

I use python because of Numpy, Scikit and Tensorflow. I don't know of any other languages with libraries as productive as these, so speeding these apps up is a big win for a lot of people. Also 20% is huge, I look forward to trying it!

20% faster is nothing. You want at least a magnitude faster to justify the cost and risk of switching. The Python language is already 30 years old, and it wasn’t even the cutting-edge in imperative language design ( Smalltalk, Lisp ) back then. It’s positively antiquated now. I’ve never understood this tunnel-vision obsession with endlessly chasing ever-diminishing returns. It’s Zawinski's Law of Software by way of G…

I agree. Almost all arguments in favor of Python is about sunk cost. Not much about the actual language is appealing compared to modern languages.

Re: Pyston v2: Faster Python

#73

Post by shoo »

nickjj wrote: post #66

It's really awesome that it's not just a speed boost but a drastic decrease in memory usage too. Going from a 230mb Flask app down to 55mb is huge if it's really a drop in replacement. If you factor in gunicorn process count, the wins are even higher because if you had 4 gunicorn processes each using 230mb but now they use 55mb, you're really going from 920mb down to 220mb of RAM. Edit: This isn't true in the end, a…

From the benchmark numbers reported in the post, pyston uses slightly more memory than cpython for the flaskblogging benchmark. switching to pyston only a win in terms of reducing memory consumption if you're using pypy, and it would be more of a win to switch to cpython

Re: Pyston v2: Faster Python

#74

Post by nickjj »

shoo wrote: post #73
nickjj wrote: post #66

It's really awesome that it's not just a speed boost but a drastic decrease in memory usage too. Going from a 230mb Flask app down to 55mb is huge if it's really a drop in replacement. If you factor in gunicorn process count, the wins are even higher because if you had 4 gunicorn processes each using 230mb but now they use 55mb, you're really going from 920mb down to 220mb of RAM. Edit: This isn't true in the end, a…

From the benchmark numbers reported in the post, pyston uses slightly more memory than cpython for the flaskblogging benchmark. switching to pyston only a win in terms of reducing memory consumption if you're using pypy, and it would be more of a win to switch to cpython

Wow thanks for the clarification.

I don't know why but I read PyPy 7.3.2 as Python 3.7 in the table. Talk about a brain auto-complete failure haha.

Re: Pyston v2: Faster Python

#75

Post by xapata »

yegle wrote: post #56
SoSoRoCoCo wrote: post #39

Ugh. At some point we need to stop using side-forks/-projects like this because then they become competing standards that pull resources away from the main projects and evolve into their own incompatible beasts. I hope they instead contribute to the main branch, instead of wandering off into NIH land.

Sure, but who to blame? There's no such thing as "Python Language Specification", and everyone is trying hard to be CPython-compatible (but ~impossible as your implementation need to be compatible with various C-written module). Should there be a "language spec", competing implementation would make the language stronger not weaker. Examples include gccgo, various Java vm, C++ compilers.

I'm pretty sure you can find the Python language specification in the form of documentation at python.org. Some things that CPython does are occasionally misconstrued as specification, but the docs usually call that out.

Re: Pyston v2: Faster Python

#76

Post by newen »

sk2020 wrote: post #44
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

Numpy has a very poor API compared to Julia, Matlab, Mathematica, R. That’s just me comparing to the ones I know. It’s a mishmash of methods and functions, in-place operations and non-modifying operations, confusing indexing and broadcasting API. There are much better things available for array manipulation.

Re: Pyston v2: Faster Python

#77

Post by kmod »

ghj wrote: post #55

I stalked the author's linkedin and notice he has competitive programming experience: https://www.topcoder.com/members/kmod/details/?track=DATA_SC... (and top 15 putnam, ICPC world finals, etc) I wonder if he would be interested in optimizing for purely algorithmic tasks? There are a lot active and successful CPython and PyPy users on https://atcoder.jp/ . For example: https://atcoder.jp/contests/practice2/submission…

All my CodeJam solutions are in Python :)

While we could certainly go in this direction, we're not planning to, because in our experience optimizations for different workloads are largely distinct, and this use case is already handled well by PyPy.

Re: Pyston v2: Faster Python

#78

Post by hhas01 »

newen wrote: post #72
hhas01 wrote: post #45

Earlier quoted context omitted.

20% faster is nothing. You want at least a magnitude faster to justify the cost and risk of switching. The Python language is already 30 years old, and it wasn’t even the cutting-edge in imperative language design ( Smalltalk, Lisp ) back then. It’s positively antiquated now. I’ve never understood this tunnel-vision obsession with endlessly chasing ever-diminishing returns. It’s Zawinski's Law of Software by way of G…

I agree. Almost all arguments in favor of Python is about sunk cost. Not much about the actual language is appealing compared to modern languages.

Ah, sunk costs. Where the future goes to die.

And I say this as a 20-year Python user myself, ’cos while it has scratched many itches and continues to do so, I am not the least bit sentimental about it. The best compliment would be to kill it with something far better, that steals all its good parts and replaces the rest.

Re: Pyston v2: Faster Python

#79

Post by ak217 »

willseth wrote: post #57
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

A different perspective is that Python leadership has been overwhelmed addressing the concerns of the enormous and growing Python community, for whom generally performance is not yet the primary concern - believe it or not. It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3, but overall it seems like the general Python community is most inte…

I agree with your overall characterization. As someone who works a lot with Python and is very familiar with its package ecosystem, I find the lack of leadership from PSF to be discouraging and upsetting. The past few Python releases have what I would characterize as cosmetic improvements while repeatedly missing opportunities to improve interpreter, packaging, and interface fundamentals. The position that CPython has to be kept simple as a reference implementation is untenable in the absence of an active collaboration or effort to produce a performance oriented implementation.

As far as I'm concerned, CPython as an interpreter technology has not advanced in the past decade - and Python has grown thanks to the data science community efforts and excellent library ecosystem you mention, not due to PSF. PSF can only miss so many opportunities before something comprehensively better starts to eclipse it.

Re: Pyston v2: Faster Python

#80

Post by ihnorton »

> A very-low-overhead JIT using DynASM

Interesting. DynASM [1] is the template assembler used in LuaJIT, so it sounds like they might be JIT'ing CPython bytecode. IIRC this is also what the first version of Pyston did. I'm curious how this is working out, both implementation and performance-wise compared to LLVM (used in Pyston v1). That could mean there is a lot of performance still on the table, at least for some kinds of code, but also a big complexity jump to get further gains.

[1] https://luajit.org/dynasm.html

Post reply on HN