Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pythran as a bridge between fast prototyping and code deployment

serge-sans-paille.github.io

Post reply on HN
Print view View original

21–27 of 27 posts

Re: Pythran as a bridge between fast prototyping and code deployment

#21

Post by dagw »

zedr wrote: post #16

Isn't this similar to the Nuitka project? http://nuitka.net/ Nuitka has fantastic Python 3 support (up to 3.7 currently).

They're similar in concept, but very different in focus. Nuitka main goal is to be 100% compatible with cpython, something which will often mean sacrificing performance compared to pythran.

Pythran main aim is to be fast, and to achieve this they're willing to only support a small subset of python.

As Nuitka's performance gets better and Pythran starts to support more and more of python, perhaps they'll converge at some point in the future.

Re: Pythran as a bridge between fast prototyping and code deployment

#22

Post by fredsanford »

Does Pythran work with things like opencv and sklearn as python modules or does code have to be written to explicitly enable them?

It feels to me like Pythran + opencv would be a killer combination since it can take 300+ lines of C++ to achieve what you can with 40ish lines of numpy, opencv and python.

Re: Pythran as a bridge between fast prototyping and code deployment

#24

Post by SubiculumCode »

kristofferc wrote: post #18

I feel that a comparison to the handwritten C++ version would make the claims a lot stronger. Making something 10x faster is not very hard if it is incredibly slow to begin with and is, on its own, fairly uninteresting. On the other hand, if the results here approcahed the speed of optimized C++ code, then this workflow makes a lot of sense.

Seems there could be a cost/benefit analysis here. Ten times faster than python might be sufficient for some applications given the potential for much faster deployment, regardless of whether handwritten C might be faster.

Re: Pythran as a bridge between fast prototyping and code deployment

#25

Post by jeanl »

SubiculumCode wrote: post #24
kristofferc wrote: post #18

I feel that a comparison to the handwritten C++ version would make the claims a lot stronger. Making something 10x faster is not very hard if it is incredibly slow to begin with and is, on its own, fairly uninteresting. On the other hand, if the results here approcahed the speed of optimized C++ code, then this workflow makes a lot of sense.

Seems there could be a cost/benefit analysis here. Ten times faster than python might be sufficient for some applications given the potential for much faster deployment, regardless of whether handwritten C might be faster.

You're absolutely right: you don't necessarily need to be within 10% of pure C++ if algo development is made far easier by using python/numpy. But it would be good to have a hand-written C++ baseline to determine where the cost/benefit point is (at least for this example).

Re: Pythran as a bridge between fast prototyping and code deployment

#26

Post by svantana »

stabbles wrote: post #8
svantana wrote: post #6

Earlier quoted context omitted.

One major issue with Julia is that it only recently reached 1.0, with a lot of breaking changes, that make a lot of libraries incompatible. Another issue is that it's not always that fast, for a recent project I never managed to exceed 100 MFLOPS, at which point I switched to C++ and got 3 GFLOPS. But the python version stalled out at 4 MFLOPS though...

Very roughly without knowing details: probably the Julia implementation can be tuned to get close to 3 GFLOPS; it's not that the language has limitations to get above this 100MFLOPS whereas in Python 4MFLOPS might potentially be the best you can get. Care to share your code and see if it can be improved upon?

I think the main performance bottleneck is that I'm adding to a submatrix. Which seems to be a big performance hit in basically all high level languages.

Re: Pythran as a bridge between fast prototyping and code deployment

#27

Post by ktta »

jeanl wrote: post #19
serge-ss-paille wrote: post #12

Earlier quoted context omitted.

> could you tell us the shortcomings of Cython In order to achieve top performance, in the context of numerical simulations, you generally end up explicity writing the loops are implicit in high-level numpy (less abstraction). Cython does not perform any high-level optimisation on the code, while Pythran does. For instance Pytrhan computes whether an array index may be negative or not, and generates wraparound only w…

For me, the biggest shortcoming: Cython does not create independent C++ code (independent of the python interpreter that is) that can be used in a separate C++ code base. My main point is that pythran makes it possible to deploy python/numpy code as C++ code.

I didn't realize you're the author of the blog post!

Thanks to both of you for the reply

Post reply on HN