Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JNumPy: Writing high-performance C extensions for Python in minutes

github.com

Post reply on HN
Print view View original

31–40 of 66 posts

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#31

Post by cycomanic »

salty_biscuits wrote: post #27
cycomanic wrote: post #24

Earlier quoted context omitted.

I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed wit…

I only had a quick look, but I would guess the slicing is what makes it slower than you would expect. Julia copies on an array slice, but you can use the @view macro or do some loops with a preallocated array to make it faster.

Thanks I'll have a look and post and update sometime later.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#32

Post by cardanome »

TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

Wait, you are using PHP for scientific computing? I would love to hear more about that part.

That could actually work out nicely with the right libraries, just haven't yet heard of anyone doing that.

I think Python is inherently difficult to optimize because it allows some dynamic trickery that PHP does not. Additionally being large and complex does not help. You can't really pull an LuaJit equivalent. There sure are still many quick wins to be made for Python performance but not sure if it will ever compete with PHP/Lua/Julia and others on the performance front.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#33

Post by webshit2 »

cycomanic wrote: post #24
adgjlsfhk1 wrote: post #5

Earlier quoted context omitted.

Do you have any source on well written julia being any slower than well written C?

I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed wit…

  wxy[:,:,k] += mu*conj(err[i,k])*X
This should probably be

  wxy[:,:,k] .+= mu*conj(err[i,k]).*X
so allocations are avoided. This doubled the speed for me, although I don't what size inputs are realistic. The @benchmark and @profile macros are good for this stuff.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#34

Post by peaky_blinder »

TekMol wrote: post #21
adgjlsfhk1 wrote: post #18

Earlier quoted context omitted.

would python being 3x faster make a difference? you'd still have so call out to an actually fast language when you want your code to be fast.

A 3x faster Python would make it about 2x slower than PHP. That is way easier to digest than the current ~5x performance penalty for using it. I would probably not consider PHP anymore if Python was 50% of its speed. I would never consider anything else than Python or PHP for web projects. Developer time is more precious than CPU time.

You can have small developer time with other technologies too (JavaScript, Go, Elixir etc) and get some additional benefits compared to PHP. Honestly I don't see the reason to use PHP today unless you have to;

1. Maintain existing PHP codebase

2. Your team only knows PHP

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#35

Post by lraxl »

anewhnaccount2 wrote: post #17
TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

It is happening: https://github.com/faster-cpython/ideas

It will yield a small speedup (Microsoft is misallocating its resources. The scientific ecosystem should be ported to .NET, with first class support for F#.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#37

Post by TekMol »

peaky_blinder wrote: post #34
TekMol wrote: post #21

Earlier quoted context omitted.

A 3x faster Python would make it about 2x slower than PHP. That is way easier to digest than the current ~5x performance penalty for using it. I would probably not consider PHP anymore if Python was 50% of its speed. I would never consider anything else than Python or PHP for web projects. Developer time is more precious than CPU time.

You can have small developer time with other technologies too (JavaScript, Go, Elixir etc) and get some additional benefits compared to PHP. Honestly I don't see the reason to use PHP today unless you have to; 1. Maintain existing PHP codebase 2. Your team only knows PHP

Except for speed, PHP has another awesome thing going for it: Statelessness.

You throw some PHP files on a server and have a working web application. As soon as you change a PHP file, the change is in effect on next reload of the page. You might be able to do the same in Python, but everybody in Python world is doing the long running thread approach, so you would be on uncharted territory.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#38

Post by pbowyer »

cardanome wrote: post #32
TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

Wait, you are using PHP for scientific computing? I would love to hear more about that part. That could actually work out nicely with the right libraries, just haven't yet heard of anyone doing that. I think Python is inherently difficult to optimize because it allows some dynamic trickery that PHP does not. Additionally being large and complex does not help. You can't really pull an LuaJit equivalent. There sure are…

> Wait, you are using PHP for scientific computing? I would love to hear more about that part.

Not OP but I wish PHP was usable for scientific computing.

If I had better C/Rust/PHP internals insane macro skills I would integrate a dataframe library (Pola.rs?) into PHP to give us the basics for manipulating data.

Plenty more to add after that for matrix multiplication and the rest but I haven't thought that far ahead!

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#39

Post by machinekob »

lraxl wrote: post #35
anewhnaccount2 wrote: post #17

Earlier quoted context omitted.

It is happening: https://github.com/faster-cpython/ideas

It will yield a small speedup ( Microsoft is misallocating its resources. The scientific ecosystem should be ported to .NET, with first class support for F#.

They already try it like 2/3 times and it didnt take off so why should it be different today?

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#40

Post by jeremycarter »

lraxl wrote: post #35
anewhnaccount2 wrote: post #17

Earlier quoted context omitted.

It is happening: https://github.com/faster-cpython/ideas

It will yield a small speedup ( Microsoft is misallocating its resources. The scientific ecosystem should be ported to .NET, with first class support for F#.

.NET would be the perfect ecosystem. And for those who love their dynamic there is support for that too.
Post reply on HN