Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JNumPy: Writing high-performance C extensions for Python in minutes

github.com

Post reply on HN
Print view View original

21–30 of 66 posts

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#21

Post by TekMol »

adgjlsfhk1 wrote: post #18
TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

would python being 3x faster make a difference? you'd still have so call out to an actually fast language when you want your code to be fast.

A 3x faster Python would make it about 2x slower than PHP.

That is way easier to digest than the current ~5x performance penalty for using it.

I would probably not consider PHP anymore if Python was 50% of its speed.

I would never consider anything else than Python or PHP for web projects. Developer time is more precious than CPU time.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#22

Post by jakobnissen »

adgjlsfhk1 wrote: post #20
jakobnissen wrote: post #19

Earlier quoted context omitted.

The point is that when adgjlsfhk1 doubts that Julia is slower than C, he does so from a background where "data" is a synonym for arrays of numbers, and "computing" means processing these arrays. Julia shines here, precisely because the people developing Julia have a similar focus. In contrast, if your performance bottlenecks are IO, parsing and hash table operations, then Julia's performance will be in par with Pytho…

on par with python is way off. Julia in these areas definitely isn't fully optimized, but the performance will be much closer to something like Java for these cases. a dict in Julia will still have 1 pointer indirect compared to the python one where all your objects have 16 bytes overhead and an extra pointer indirection.

Python's dicts are way more optimised and have an implementation closer to Dictionaries.jl.

Anyway, I think it might be a good exercise for me to make a repo with a few test cases of where I think Julia could use some optimisations, and compare to Rust and Python.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#23

Post by TekMol »

anewhnaccount2 wrote: post #17
TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

It is happening: https://github.com/faster-cpython/ideas

It seems to indicate a 30% performance boost?

That would be a step into the right direction.

6 more 30% improvements and it is on par with PHP.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#24

Post by cycomanic »

adgjlsfhk1 wrote: post #5
cycomanic wrote: post #3

Unless I'm misunderstanding, this seems to be writing python extensions in Julia not C. So quite a misleading title. Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases). I really wish the Julia community would stop oversel…

Do you have any source on well written julia being any slower than well written C?

I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed with the ease of Python. That's actually my main gripe with Julia, the community advertises it as something that it isn't. I think otherwise some really cool stuff is happening in Julia Land.

[1] https://jochenschroeder.com/blog/articles/DSP_with_Python2/

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#25

Post by cycomanic »

pizza wrote: post #6
cycomanic wrote: post #3

Unless I'm misunderstanding, this seems to be writing python extensions in Julia not C. So quite a misleading title. Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases). I really wish the Julia community would stop oversel…

This is why it's a C extension: https://github.com/Suzhou-Tongyuan/jnumpy/blob/main/TyPython... The title is correct because it is a Julia numpy interface. Inside it seems is another project, TyPython, that provides an efficient Julia-Python-Numpy bridge, via C extensions.

I would argue that it might technically be correct (and admittedly that's the best type of correct) in the strict sense, it is also misleading. If I read that I'm writing C extensions in minutes I at least expect that the extension compiles to C (which the Julia code doesn't).

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#26

Post by majoe »

metadat wrote: post #2

(Writing Python extensions in Julia) ..which could be a heckuva lot friendlier and often safer than C! The requisite python boilerplate is pretty ugly, though: init_jl() include_src('example.jl', __file__) exec_julia('example.init()') I also don't understand how jl_mat_mul gets mapped to mat_mul. jl_mat_mul = Pyfunc(jl_mat_mul) Huh?

I had a look at the demo folder, that's just a typo. It should be:

jl_mat_mul = Pyfunc(mat_mul)

which makes a lot more sense.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#27

Post by salty_biscuits »

cycomanic wrote: post #24
adgjlsfhk1 wrote: post #5

Earlier quoted context omitted.

Do you have any source on well written julia being any slower than well written C?

I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed wit…

I only had a quick look, but I would guess the slicing is what makes it slower than you would expect. Julia copies on an array slice, but you can use the @view macro or do some loops with a preallocated array to make it faster.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#28

Post by majoe »

Coincidentally I used PyJulia last week to call some numerical code I translated from Python/Numpy to Julia. The actual translation was straightforward and done in under one hour, but I needed an additional day for fine tuning and profiling to actually make it fast.

Does anyone know, how JNumPy compares to PyJulia? Calling my Julia code via PyJulia seems to increase the startup time of Julia even more and it would be great, if JNumpy would do better here.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#30

Post by meta2meta »

cycomanic wrote: post #24
adgjlsfhk1 wrote: post #5

Earlier quoted context omitted.

Do you have any source on well written julia being any slower than well written C?

I did a comparison of Julia vs numpy, cython and pythran [1] some time ago, for a typical dsp routine we use in our work, and Julia was quite a bit slower than the alternatives. Now I'm by no means a Julia expert so I might have missed an optimisation opportunity (although I posted this and nobody could point to something obvious) , however the whole advertisement behind Julia is that one gets essentially C speed wit…

You should run Julia code directly to benchmark instead of calling it in Python through pyjulia.
Post reply on HN