Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyston v2: Faster Python

blog.pyston.org

Post reply on HN
Print view View original

151–160 of 211 posts

Re: Pyston v2: Faster Python

#151

Post by hansvm »

newen wrote: post #76
sk2020 wrote: post #44

Earlier quoted context omitted.

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

Numpy has a very poor API compared to Julia, Matlab, Mathematica, R. That’s just me comparing to the ones I know. It’s a mishmash of methods and functions, in-place operations and non-modifying operations, confusing indexing and broadcasting API. There are much better things available for array manipulation.

For all of numpy's faults I find it odd that you choose its indexing and broadcasting API as the thing that's confusing and worse than equivalents in Mathematica and R. I can't speak to Julia or Matlab, but I find working with arrays in the other two is like pulling teeth.

Re: Pyston v2: Faster Python

#152

Post by pansa2 »

throwaway894345 wrote: post #58
mrtranscendence wrote: post #49

Earlier quoted context omitted.

> Meanwhile, there are many other languages which are not only performant, but which are rapidly encroaching on Python's historically unique(ish) "easiness" What are those languages? I may have a blindspot, but the languages that get enough buzz for me to notice are either not competing with Python in important dimensions (e.g. Rust) or have a narrower focus (e.g. Julia). Elixir maybe? JavaScript and its derivatives?…

> I'd love to find a viable competitor to Python that's strictly better than it. I strongly recommend Go as a better Python. Personally, I think it's easier to write than Python (although people who care very little about correctness will be bothered a bit by the type checker), and the tooling is many times better (single-binary deployments, great dependency management, etc are awesome). Also, the performance is abou…

> single-binary deployments

More than performance (pure Python can be too slow, but NumPy is usually fast enough), this is what I miss when using Python.

There's currently no good solution for bundling Python code and the interpreter into a single binary. PyInstaller works, but the resulting binaries write lots of files to `/tmp` every time they run, which is a hack that leads to long startup times. PyOxidizer avoids this, but includes the entire standard library in every binary, making them too large by an order of magnitude.

Re: Pyston v2: Faster Python

#154

Post by phonon »

radlad wrote: post #147
musicale wrote: post #132

Earlier quoted context omitted.

> It's probably fair to suggest PSF has stumbled in executing some of their goals, most notably and publicly the transition to v3 In a recent post linked on HN, Steve Yegge basically nailed it: > How much new software was written in something other than Python, which might have been written in Python if Guido hadn’t burned everyone’s house down? It’s hard to say, but I can tell you, it hasn’t been good for Python. It…

Link?

https://medium.com/@steve.yegge/dear-google-cloud-your-depre...

Re: Pyston v2: Faster Python

#155

Post by ihnorton »

willseth wrote: post #137
ihnorton wrote: post #121

Earlier quoted context omitted.

In the context of pandas, 3 GB of (raw, uncompressed) data could easily require 30 GB of RAM, and that kind of overhead adds up quickly.

Pandas is not some mysterious black box. If you need predictable runtime performance or bounded memory usage, you have to figure it out. Pandas doesn't inherently have a staggering or unpredictable amount of overhead, given that it's a statistical analysis package. There are ways to mitigate Pandas memory usage (10x is a sign that something has gone very horribly wrong), and sometimes Pandas is simply the wrong tool…

10x reflects both experience and expert recommendations. You may recognize the author [1]:

> Nowadays, my rule of thumb for pandas is that you should have 5 to 10 times as much RAM as the size of your dataset

[1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/

Re: Pyston v2: Faster Python

#156

Post by chrisseaton »

omginternets wrote: post #133
chrisseaton wrote: post #126

Earlier quoted context omitted.

> incompatibilities, etc But it's compatible. It's a drop-in replacement.

Serious question: do you have experience with “drop in replacements” for reference implementations? They’re rarely 100% compatible. Especially when the reference implementation isn’t formally specified. Moreover, you continue to cherry pick, neglecting the other arguments you’ve been presented. It sounds like you’ve already made up your mind about this.

Other arguments are ‘it’s not worth it’ or ‘it’s not likely to succeed’ but nobody is making you do the work!

Re: Pyston v2: Faster Python

#157

Post by omginternets »

chrisseaton wrote: post #156
omginternets wrote: post #133

Earlier quoted context omitted.

Serious question: do you have experience with “drop in replacements” for reference implementations? They’re rarely 100% compatible. Especially when the reference implementation isn’t formally specified. Moreover, you continue to cherry pick, neglecting the other arguments you’ve been presented. It sounds like you’ve already made up your mind about this.

Other arguments are ‘it’s not worth it’ or ‘it’s not likely to succeed’ but nobody is making you do the work!

Hmm no, they’re really not. I’ll leave you to reread the tread.

Re: Pyston v2: Faster Python

#158

Post by jpeloquin »

throwaway894345 wrote: post #51
sk2020 wrote: post #44

Earlier quoted context omitted.

Scipy has a poor performance ceiling? Numpy has a poor API? Compared to what? Eigen? Whatever the Scala guys use? That sounds kind of silly to me, especially when hardly anyone is actually CPU-bound, anyway.

WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention). WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The ref…

> there's nothing inherent to any particular domain that demands this kind of 'magical' API

Plotting seems to tend towards magic because plots are basically art, with all the desire for aesthetic customization that applies, and it's a very common task so users also want brevity (magic). The result is a plot() function with a gazillion options hidden behind keyword arguments.

I agree that matplotlib has a sprawling interface, and this can be annoying, but I'm still not sure what "guess the caller's intent by inspecting the types of the arguments" means. Sure, the functions have multiple call signatures, but that's not exactly unusual in libraries or languages. I don't understand the context that brings guesswork into the picture. Skimming the manual—are you using the data keyword argument and hitting the `plot('n', 'o', data=obj)` ambiguity [0]? Or calling plot through `pyplot.plot` &c. (which rely on state) instead of `Axes.plot` &c.?

Asking because if there's an interface trap I'm unaware of I'd like to learn about it before walking into it blindly.

Pandas I sort of agree with; I personally find it harder to remember how to use pandas than dplyr, despite using pandas more often and spending more time reading the pandas documentation. I also find it inconvenient to represent missing values in Pandas (`None` and `NaN` are overloaded, and `None` forces the `object` dtype). But maybe the problem is on my end.

[0] https://matplotlib.org/3.3.2/api/_as_gen/matplotlib.pyplot.p...

Re: Pyston v2: Faster Python

#159

Post by xvilka »

There is another way. Since Pyston isn't generally compatible with the original Python, why not to use faster languages? Go for network services and console tools. Julia for AI applications, data and computational science.

Re: Pyston v2: Faster Python

#160

Post by xvilka »

solidasparagus wrote: post #143
throwaway894345 wrote: post #58

Earlier quoted context omitted.

> I'd love to find a viable competitor to Python that's strictly better than it. I strongly recommend Go as a better Python. Personally, I think it's easier to write than Python (although people who care very little about correctness will be bothered a bit by the type checker), and the tooling is many times better (single-binary deployments, great dependency management, etc are awesome). Also, the performance is abou…

Go and Python have pretty minimal overlap IMO. If you are using Python for anything other than a server or CLI, Golang is not a very good replacement for Python. Some of Python's strengths that I work with regularly are dynamism, easy data exploration, visualizations, succinct & customizable syntax, extremely strong data science libraries, REPL/Jupyter, C bindings, easy to use packaging solution via PyPi (bet some pe…

Julia covers your use cases and overwhelmingly fast.
Post reply on HN