Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

131–140 of 211 posts

Re: Pyston v2: Faster Python

#131

Post by ak217 »

mrtranscendence wrote: post #104
ak217 wrote: post #82

Earlier quoted context omitted.

Thanks - Mark Shannon is a core committer, right? So hopefully he has standing to get this done. The plan looks very high level at this point, but it looks like Mark is an expert in interpreter VM and JIT technologies. All I can hope for is that he doesn't get blocked by the "keep cpython simple" obstructionism.

He thinks (they think?) the project will need to be funded to the tune of $2M. That seems like a hefty sum to me, though maybe it's doable.

It's not that hefty once they pitch this improvement to the many wealthy VC-funded companies whose business and data science divisions depend on Python.

Re: Pyston v2: Faster Python

#132

Post by musicale »

willseth wrote: post #57
throwaway894345 wrote: post #29

Earlier quoted context omitted.

This is why I see little hope for Python, which is to say that while I'm sure it will continue to have a large following for many years a la C, C++, etc, I don't have hope for it being an exciting language or one that is particularly productive. Python already has performance and packaging problems which don't seem to be easily divorced from CPython, since virtually the whole reference implementation is depended upon…

A different perspective is that Python leadership has been overwhelmed addressing the concerns of the enormous and growing Python community, for whom generally performance is not yet the primary concern - believe it or not. It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3, but overall it seems like the general Python community is most inte…

> It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3

In a recent post linked on HN, Steve Yegge basically nailed it:

> How much new software was written in something other than Python, which might have been written in Python if Guido hadn’t burned everyone’s house down? It’s hard to say, but I can tell you, it hasn’t been good for Python. It’s a huge mess and everyone is miserable.

Note to future language maintainers: don't burn everyone's house down.

Re: Pyston v2: Faster Python

#133

Post by omginternets »

chrisseaton wrote: post #126
omginternets wrote: post #123

Earlier quoted context omitted.

A fragmented ecosystem, incompatibilities, etc. And more importantly, your original question isn't asking about forks, specifically. It was asking why someone might oppose performance work.

> incompatibilities, etc But it's compatible. It's a drop-in replacement.

Serious question: do you have experience with “drop in replacements” for reference implementations? They’re rarely 100% compatible. Especially when the reference implementation isn’t formally specified.

Moreover, you continue to cherry pick, neglecting the other arguments you’ve been presented. It sounds like you’ve already made up your mind about this.

Re: Pyston v2: Faster Python

#134

Post by toolslive »

jashmatthews wrote: post #97
toolslive wrote: post #36

Earlier quoted context omitted.

it's _still_ essentially the same switch(opcode) based interpreter it was 20 years ago. no threading, no super instructions, no jitting, nothing. https://github.com/python/cpython/blob/master/Python/ceval.c...

Direct threading is in here https://github.com/python/cpython/blob/0564aafb71a153dd0aca4...

ha! you're right. so some optimizations did happen while I wasn't looking ;)

Re: Pyston v2: Faster Python

#135

Post by dragonwriter »

hhas01 wrote: post #106
dragonwriter wrote: post #95

Earlier quoted context omitted.

> Almost all arguments in favor of Python is about sunk cost. I think you are confusing ecosystem and other established advantages with sunk costs, they are different things. It's true that (from the perspective of the people who built them), those advantages are the products of sunk costs, but the argument is about the ongoing value delivered, not the sunk cost involved in delivering it. > Not much about the actual…

“the argument is about the ongoing value delivered” Which is an admirable sentiment… but the title of this thread is not “Python: still doing useful work” but “Python: now 20% faster”, and being ridiculously self-congratulatory about this when the correct response is to laugh at the silly pointless frivolity of it. Trying to make Python fast is a fool’s errand, because Python is slow by design . A useful argument wou…

> Which is an admirable sentiment… but the title of this thread is not “Python: still doing useful work” but “Python: now 20% faster”

No, it's actually, “Pyston v2: 20% faster Python". But...so what?

> and being ridiculously self-congratulatory about this when the correct response is to laugh at the silly pointless frivolity of it

For the same reasons Python is often a valuable choice, a faster Python is a valuable option.

> Trying to make Python fast is a fool’s errand, because Python is slow by design.

Python is not slow by design, though it's slow because it's not fast by design. But that doesn't mean it's not useful to have a faster Python, only that there are likely to be limits and trade-offs involved in doing that.

> A useful argument would be that Python is faster overall at solving various real-world problems than current alternatives; but that’s not the popular argument being made

Yes, actually, it is. It's not the message of the Pyston v2 release blog entry, but then that blog entry isn't making an argument in the debate that you seem to want everything to be about.

Re: Pyston v2: Faster Python

#137

Post by willseth »

ihnorton wrote: post #121
willseth wrote: post #61

Earlier quoted context omitted.

"Based on prototypes I [spent a limited amount of time on and didn't research better methods], I'm confident..."

In the context of pandas, 3 GB of (raw, uncompressed) data could easily require 30 GB of RAM, and that kind of overhead adds up quickly.

Pandas is not some mysterious black box. If you need predictable runtime performance or bounded memory usage, you have to figure it out. Pandas doesn't inherently have a staggering or unpredictable amount of overhead, given that it's a statistical analysis package. There are ways to mitigate Pandas memory usage (10x is a sign that something has gone very horribly wrong), and sometimes Pandas is simply the wrong tool for the job.

Re: Pyston v2: Faster Python

#138

Post by kzrdude »

bearzoo wrote: post #116
throwaway894345 wrote: post #51

Earlier quoted context omitted.

WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention). WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The ref…

many scientific computing applications are considered to be bounded by io

rather famously, one needs an intense operation like matrix multiplication to get cpu bound (an operation that has many enough arithmetic operations per data element, for I/O to not dominate).

Re: Pyston v2: Faster Python

#139

Post by kmod »

ihnorton wrote: post #80

> A very-low-overhead JIT using DynASM Interesting. DynASM [1] is the template assembler used in LuaJIT, so it sounds like they might be JIT'ing CPython bytecode. IIRC this is also what the first version of Pyston did. I'm curious how this is working out, both implementation and performance-wise compared to LLVM (used in Pyston v1). That could mean there is a lot of performance still on the table, at least for some k…

We started with an LLVM-based JIT in Pyston v1. Our experience with the two jits is that it's very nice that DynASM is ~2 orders of magnitude faster, and also that we have not been able to extract enough high-level knowledge to make a powerful JIT like LLVM worth it. We do use LLVM elsewhere in our build process, and we hope to write some future blogposts about all of this.

Re: Pyston v2: Faster Python

#140

Post by kmod »

ghj wrote: post #114
kmod wrote: post #77

Earlier quoted context omitted.

All my CodeJam solutions are in Python :) While we could certainly go in this direction, we're not planning to, because in our experience optimizations for different workloads are largely distinct, and this use case is already handled well by PyPy.

Isn't this use case the scientific computing use case? That's a fairly large part of the ecosystem to give up on! I think it's still a relatively low effort way (just need to write a scraper) to create a benchmark on a diverse set of algorithmic tasks that have clearcut criteria on AC/TLE/WA. PyPy is often 10x faster than cpython on these problems (and just 2x slower than equivalent C++ solution) so it will be a much…

This is just a comment on my personal use of Python for competitive programming: I've never used numpy for competitive programming or thought that it would be a good tool for that. PyPy seems like a great solution for the highly-numerical algorithms that these contests tend to lead to.

So I would not call this "scientific computing". Personally I consider competitive programming to be it's own use case.

And as much as we want to improve scientific computing in Python, it's very hard since the work is done in C. Our current hope is to help mixed workloads, such as doing a decent amount of data-preprocessing in Python before handing off to C code.

Post reply on HN