Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

51–60 of 211 posts

Re: Pyston v2: Faster Python

#51

Post by throwaway894345 »

sk2020 wrote: post #44
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention).

WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The referent isn't "some other scientific computing API" (although I'm sure there are some sane scientific computing APIs), but rather "other APIs in general" since there's nothing inherent to any particular domain that demands this kind of 'magical' API.

WRT 'hardly anyone is CPU bound'--the context is numeric computing; what are people bound by if not CPU? I've seen several projects where web endpoints were timing out while grinding in Pandas, largely because there weren't good options for taking advantage of multiple processors. Based on prototypes I did, I'm confident that other languages could serve those requests in single-digit seconds if not sub-second.

Re: Pyston v2: Faster Python

#52

Post by imtringued »

SoSoRoCoCo wrote: post #39

Ugh. At some point we need to stop using side-forks/-projects like this because then they become competing standards that pull resources away from the main projects and evolve into their own incompatible beasts. I hope they instead contribute to the main branch, instead of wandering off into NIH land.

The solution to that is not to stick your head into sand, it's to merge the desired changes.

Re: Pyston v2: Faster Python

#53

Post by hitekker »

saurik wrote: post #38
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

100% agree. To a very real extent, Go probably only exists at all because Google poured an infinite amount of effort into Unladen Swallow--a project designed to remove the GIL from Python 2 to make it more usable for concurrent programs and add an LLVM-based JIT compiler to improve performance--and Python's response was to not only not merge it, but to break the entire Python ecosystem for a decade by forking the lan…

Huh, I was wondering why Unladen Swallow faded rapidly since 2009 and why Python seems secondary at Google compared to Java and Go.

Silence and distance seems to be a common echo of failed projects.

Re: Pyston v2: Faster Python

#54

Post by rhencke »

saurik wrote: post #38
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

100% agree. To a very real extent, Go probably only exists at all because Google poured an infinite amount of effort into Unladen Swallow--a project designed to remove the GIL from Python 2 to make it more usable for concurrent programs and add an LLVM-based JIT compiler to improve performance--and Python's response was to not only not merge it, but to break the entire Python ecosystem for a decade by forking the lan…

Google did not hire Rob Pike with Go in mind. Go did not even come around as an idea until Pike had worked at Google for a while. [1]

Don't forget Rob Pike created Sawzall at Google first (2005). [2]

[1] https://golang.org/doc/faq#history [2] https://research.google/pubs/pub61/

Re: Pyston v2: Faster Python

#55

Post by ghj »

I stalked the author's linkedin and notice he has competitive programming experience: https://www.topcoder.com/members/kmod/details/?track=DATA_SC... (and top 15 putnam, ICPC world finals, etc)

I wonder if he would be interested in optimizing for purely algorithmic tasks?

There are a lot active and successful CPython and PyPy users on https://atcoder.jp/. For example:

https://atcoder.jp/contests/practice2/submissions?f.Task=&f.... (the user "maspy" is rated at 2750 using only cpython!!!)

https://atcoder.jp/contests/practice2/submissions?f.Task=&f.... (though pypy is more practical)

I am linking to atcoder because their testing data is public so you can rerun contestants solutions using both pyston/cpython/pypy for benchmarking purposes: https://www.dropbox.com/sh/arnpe0ef5wds8cv/AAAk_SECQ2Nc6SVGi...

Right now, other than a handful of people who figured out how to make numba's jit work, only pypy is viable for competitive programming. I wonder if you can do better than pypy?

There are also a few red coders on codeforces.com who mostly use pypy (cpython is completely unviable there because numpy and numba is not installed)

https://codeforces.com/submissions/pajenegod

https://codeforces.com/submissions/conqueror_of_tourist

But codeforces' test cases aren't public anyway so it's not as relevant.

Re: Pyston v2: Faster Python

#56

Post by yegle »

SoSoRoCoCo wrote: post #39

Ugh. At some point we need to stop using side-forks/-projects like this because then they become competing standards that pull resources away from the main projects and evolve into their own incompatible beasts. I hope they instead contribute to the main branch, instead of wandering off into NIH land.

Sure, but who to blame? There's no such thing as "Python Language Specification", and everyone is trying hard to be CPython-compatible (but ~impossible as your implementation need to be compatible with various C-written module).

Should there be a "language spec", competing implementation would make the language stronger not weaker. Examples include gccgo, various Java vm, C++ compilers.

Re: Pyston v2: Faster Python

#57

Post by willseth »

throwaway894345 wrote: post #29
rrdharan wrote: post #17

Earlier quoted context omitted.

Python is a welcoming community, but they are decidedly not welcoming of contributions that significantly increase the complexity of the CPython reference implementation. This has been discussed extensively: https://news.ycombinator.com/item?id=11125769

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

A different perspective is that Python leadership has been overwhelmed addressing the concerns of the enormous and growing Python community, for whom generally performance is not yet the primary concern - believe it or not. It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3, but overall it seems like the general Python community is most interested in language and ecosystem features, while performance-critical workloads are already being addressed in a number of projects (not just PyPy, but Numba, Cython, and others).

Data science may be at the heart of Python's strengths, but it's simply not accurate to suggest Python (whoever that is!) is resting on its laurels, as evidenced by the very active, albeit sprawling, package ecosystem. And while the CPython devs seem to have found their stride in 3.x releases.

It's also inaccurate to suggest numeric computing in Python has a "low performance ceiling," considering you can get near-C performance via JIT or AOT using a package like Numba, and in most cases it's not even necessary because of Numpy and the many other highly optimized compute packages that can do most of the heavy lifting.

I think the main draw of Python is not just that the syntax and language features are approachable, but that the package ecosystem is so broad and active, you are likely to make lighter work of the same job done in another language. I think to displace Python you would have to displace the package ecosystem, which seems as big and broad as it's ever been.

Re: Pyston v2: Faster Python

#58

Post by throwaway894345 »

mrtranscendence wrote: post #49
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

> Meanwhile, there are many other languages which are not only performant, but which are rapidly encroaching on Python's historically unique(ish) "easiness" What are those languages? I may have a blindspot, but the languages that get enough buzz for me to notice are either not competing with Python in important dimensions (e.g. Rust) or have a narrower focus (e.g. Julia). Elixir maybe? JavaScript and its derivatives?…

> I'd love to find a viable competitor to Python that's strictly better than it.

I strongly recommend Go as a better Python. Personally, I think it's easier to write than Python (although people who care very little about correctness will be bothered a bit by the type checker), and the tooling is many times better (single-binary deployments, great dependency management, etc are awesome). Also, the performance is about 100-1000 times better for serial execution, and Go's goroutines allow you to take advantage of multiple cores much more easily than with Python.

JavaScript and TypeScript are similarly easy-to-use, performant languages with a better-than-Python tooling story. I've also heard similar things about Elixir, Closure, and Kotlin.

Re: Pyston v2: Faster Python

#60

Post by vosper »

throwaway894345 wrote: post #58
mrtranscendence wrote: post #49

Earlier quoted context omitted.

> Meanwhile, there are many other languages which are not only performant, but which are rapidly encroaching on Python's historically unique(ish) "easiness" What are those languages? I may have a blindspot, but the languages that get enough buzz for me to notice are either not competing with Python in important dimensions (e.g. Rust) or have a narrower focus (e.g. Julia). Elixir maybe? JavaScript and its derivatives?…

> I'd love to find a viable competitor to Python that's strictly better than it. I strongly recommend Go as a better Python. Personally, I think it's easier to write than Python (although people who care very little about correctness will be bothered a bit by the type checker), and the tooling is many times better (single-binary deployments, great dependency management, etc are awesome). Also, the performance is abou…

What's Go like for REPL-driven / exploratory development? That's mostly how I use Python.
Post reply on HN