Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

91–100 of 211 posts

Re: Pyston v2: Faster Python

#91

Post by willseth »

ak217 wrote: post #79
willseth wrote: post #57

Earlier quoted context omitted.

A different perspective is that Python leadership has been overwhelmed addressing the concerns of the enormous and growing Python community, for whom generally performance is not yet the primary concern - believe it or not. It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3, but overall it seems like the general Python community is most inte…

I agree with your overall characterization. As someone who works a lot with Python and is very familiar with its package ecosystem, I find the lack of leadership from PSF to be discouraging and upsetting. The past few Python releases have what I would characterize as cosmetic improvements while repeatedly missing opportunities to improve interpreter, packaging, and interface fundamentals. The position that CPython ha…

I do get the impression that position is not the overwhelming consensus among CPython core devs (nor does it really make sense, given the huge dependency of many packages to the C API), whether or not it is what PSF may have publicly communicated. There are a few glimmers that interpreter improvements are on the horizon, with some proposals like subinterpreters and GIL mitigation getting a lot of attention, all which (to my knowledge) are necessary and prerequisite to serious, bold performance improvements.

I agree performance is important, and I think we have reason to be optimistic, but with the understanding that that level of improvement, even if currently underway, will just take a lot of effort and time. Meanwhile, as someone fairly new to Python and working on several performance critical pieces, I've been pretty impressed with what you can do with the current compute packages (after taking months to work my way through most of them).

Re: Pyston v2: Faster Python

#92

Post by jnwatson »

The_rationalist wrote: post #40

Interesting to see that it significantly outperform pypy on some metrics, but wouldn't have it been better to allocate the human resources towards pypy instead of a duplicated effort?

They are completely different, incompatible approaches. Pypy in particular has a C compatibility issue.

Re: Pyston v2: Faster Python

#93

Post by kapilvt »

soperj wrote: post #16
EdSchouten wrote: post #9

Not that it's necessarily a bad thing, but it looks like the Pyston project isn't getting a lot of updates: https://github.com/pyston/pyston/commits/v2.0

Project says: A faster and highly-compatible implementation of the Python programming language. The code here is out of date, please follow our blog

where is the code? the repo looks relatively untouched, the blog directs to this repo for filing issues. https://github.com/pyston/pyston

Re: Pyston v2: Faster Python

#94

Post by mrtranscendence »

throwaway894345 wrote: post #51
sk2020 wrote: post #44

Earlier quoted context omitted.

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention). WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The ref…

You're getting some pushback, but I tend to agree with you on matplotlib and pandas. Great libraries are designed so that you can get a feel for them and -- with practice -- use them intuitively. Even after years of (admittedly light) use I still find pandas' multi-indexes confusing, and I always have to look up the best of myriad ways to do something in matplotlib. In comparison, R's dplyr and ggplot have stuck with me even ages after giving up day-to-day use of R.

Re: Pyston v2: Faster Python

#95

Post by dragonwriter »

newen wrote: post #72
hhas01 wrote: post #45

Earlier quoted context omitted.

20% faster is nothing. You want at least a magnitude faster to justify the cost and risk of switching. The Python language is already 30 years old, and it wasn’t even the cutting-edge in imperative language design ( Smalltalk, Lisp ) back then. It’s positively antiquated now. I’ve never understood this tunnel-vision obsession with endlessly chasing ever-diminishing returns. It’s Zawinski's Law of Software by way of G…

I agree. Almost all arguments in favor of Python is about sunk cost. Not much about the actual language is appealing compared to modern languages.

> Almost all arguments in favor of Python is about sunk cost.

I think you are confusing ecosystem and other established advantages with sunk costs, they are different things. It's true that (from the perspective of the people who built them), those advantages are the products of sunk costs, but the argument is about the ongoing value delivered, not the sunk cost involved in delivering it.

> Not much about the actual language is appealing compared to modern languages.

Even if that was true, many of the actual languages competing with Python are less modern by any measure, and in any case so what? Does it matter when choosing a langauge if an advantage is produced by the abstract design of a language, it's ecosystem, or the peculiarities of the available implementations? Advantages are advantages, value is value.

Sure, if you are considering how to promote a “modern” language against Python, it's important to distinguish whether your current barrier is the design of your language or Python’s ecosystem to know how to direct your efforts, but if you aren't a tool evangelist and instead are choosing a language for a project, I don't see that it matters why Python is a net advantage, as long as it is.

Re: Pyston v2: Faster Python

#96

Post by soperj »

kapilvt wrote: post #93
soperj wrote: post #16

Earlier quoted context omitted.

Project says: A faster and highly-compatible implementation of the Python programming language. The code here is out of date, please follow our blog

where is the code? the repo looks relatively untouched, the blog directs to this repo for filing issues. https://github.com/pyston/pyston

I think it's private. Looks like a for profit fork.

Re: Pyston v2: Faster Python

#97

Post by jashmatthews »

toolslive wrote: post #36
rrdharan wrote: post #17

Earlier quoted context omitted.

Python is a welcoming community, but they are decidedly not welcoming of contributions that significantly increase the complexity of the CPython reference implementation. This has been discussed extensively: https://news.ycombinator.com/item?id=11125769

it's _still_ essentially the same switch(opcode) based interpreter it was 20 years ago. no threading, no super instructions, no jitting, nothing. https://github.com/python/cpython/blob/master/Python/ceval.c...

Direct threading is in here https://github.com/python/cpython/blob/0564aafb71a153dd0aca4...

Re: Pyston v2: Faster Python

#98

Post by yegle »

xapata wrote: post #75
yegle wrote: post #56

Earlier quoted context omitted.

Sure, but who to blame? There's no such thing as "Python Language Specification", and everyone is trying hard to be CPython-compatible (but ~impossible as your implementation need to be compatible with various C-written module). Should there be a "language spec", competing implementation would make the language stronger not weaker. Examples include gccgo, various Java vm, C++ compilers.

I'm pretty sure you can find the Python language specification in the form of documentation at python.org. Some things that CPython does are occasionally misconstrued as specification, but the docs usually call that out.

The closest thing to a "language spec" is https://docs.python.org/3/reference/index.html, which still contains a lot of CPython specific/internal implementations (e.g. https://docs.python.org/3/reference/datamodel.html?highlight...).

This, and combined with https://www.hyrumslaw.com/, makes this "Python Language Reference" a "CPython Language Reference".

Re: Pyston v2: Faster Python

#99

Post by samcolvin »

jedberg wrote: post #10

I'm not versed in the details of the politics of CPython, but why did this project fork instead of just contributing to CPython? Is CPython really slow in integrating community contributions? Edit: I read the blog post closer, and found this: "Our plan is to open-source the code in the future, but since compiler projects are expensive and we no longer have benevolent corporate sponsorship, it is currently closed-sour…

Victor Stinner (Python core developer focused on performance) has a great speech about this question:

There are a lot of reasons actually

- Performance limited by old CPython design. If you fork it you have to deal with all the legacy code.

- CPython is limited to 1 thread because of the GIL.

- Specific memory allocators, C structures, reference counting, specific garbage collector etc.

You can find that video in here: https://youtu.be/TXRPCZ7Nmh4

Re: Pyston v2: Faster Python

#100

Post by jnwatson »

rrdharan wrote: post #17
dwheeler wrote: post #13

Earlier quoted context omitted.

As far as I know CPython is quite welcoming. However, per the blog post, this version of Pyston is closed source. So CPython won't be interested, and many others won't be interested either.

Python is a welcoming community, but they are decidedly not welcoming of contributions that significantly increase the complexity of the CPython reference implementation. This has been discussed extensively: https://news.ycombinator.com/item?id=11125769

That post was 4 years ago. Even though I haven't heard explicitly, certainly some of the recent comment from core developer talk about performance techniques that would definitely make things more complicated.

There's new boss(es) over the Python project, and things might be changing in the near future.

Post reply on HN