Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JNumPy: Writing high-performance C extensions for Python in minutes

github.com

Post reply on HN
Print view View original

11–20 of 66 posts

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#11

Post by jkrubin »

jakobnissen wrote: post #10
adgjlsfhk1 wrote: post #5

Earlier quoted context omitted.

Do you have any source on well written julia being any slower than well written C?

Don't be unreasonable. Numerical Julia code is just as fast, precisely because it's easy to design efficient high-level APIs and let the compiler optimise it. Try timing Julia's IO, or it's string processing, or it's hash tables, or its array allocation or it's GC - you know, nearly everything other that mathematical operations on floats or ints. Nearly everything leaves performance on the table. Sure Julia is fast,…

What’s your point?

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#12

Post by anewhnaccount2 »

cycomanic wrote: post #3

Unless I'm misunderstanding, this seems to be writing python extensions in Julia not C. So quite a misleading title. Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases). I really wish the Julia community would stop oversel…

> Also why should I pick this over cython, pythran or numba?

One scenario would be that there is code already written in Julia, which you would like to expose to a broader audience. Although overall Julia's ecosystem is smaller than Python's, there are some niches within which it is highly developed. Library code accessible from the alternatives you listed is mainly limited to existing Python and C code. See the demo with `ParallelKMeans.jl`, which wraps a high performance Julia library performing K-means clustering, which could be a useful addition to the Python ecosystem.

Another scenario is that you want to write some custom code which makes use of the Julia ecosystem and then expose it to Python.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#13

Post by anewhnaccount2 »

pizza wrote: post #6
cycomanic wrote: post #3

Unless I'm misunderstanding, this seems to be writing python extensions in Julia not C. So quite a misleading title. Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases). I really wish the Julia community would stop oversel…

This is why it's a C extension: https://github.com/Suzhou-Tongyuan/jnumpy/blob/main/TyPython... The title is correct because it is a Julia numpy interface. Inside it seems is another project, TyPython, that provides an efficient Julia-Python-Numpy bridge, via C extensions.

Agreed that the title is technically accurate. Julia should also be in the title, but this is possibly an oversight. It seems a bit unnecessary for the GP to assume that this is a bad faith attempt to mislead and insert Julia where it is not welcome rather than a potentially useful extra tool for writing high performance code. Note that the author/poster is probably not a native English speaker.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#14

Post by TekMol »

Python itself has to become faster.

For the love of god. Make it as fast as PHP.

My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness.

Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do!

The result would be heaven.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#15

Post by anewhnaccount2 »

jkrubin wrote: post #11
jakobnissen wrote: post #10

Earlier quoted context omitted.

Don't be unreasonable. Numerical Julia code is just as fast, precisely because it's easy to design efficient high-level APIs and let the compiler optimise it. Try timing Julia's IO, or it's string processing, or it's hash tables, or its array allocation or it's GC - you know, nearly everything other that mathematical operations on floats or ints. Nearly everything leaves performance on the table. Sure Julia is fast,…

What’s your point?

The point is that these things (string processing/hash tables) are typically done using custom/tuned code in C and so would be faster than the generic, and not particularly carefully tuned Julia implementations.

However, given this library seems to be about passing NumPy arrays to/from Julia, the application is almost certainly to expose numerical stuff from Julia to Python. Basically, the use case is similar to Numba, but it works by producing a C-extension from a Julia system image (if I understand correctly). The main advantage over something like Numba would be access to the Julia ecosystem.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#16

Post by oefrha »

cycomanic wrote: post #3

Unless I'm misunderstanding, this seems to be writing python extensions in Julia not C. So quite a misleading title. Also why should I pick this over cython, pythran or numba? with those I don't need to learn how to optimise another language (and no just writing Julia code does not necessarily get you large speed ups and certainly not the speed of C in many cases). I really wish the Julia community would stop oversel…

Title says “C extension” which is accurate, not “extension written in C”. You can write C extensions for CPython in non-C languages with a bridge, e.g. PyO3 for Rust, but the extension ultimately has to use the C API and ABI.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#17

Post by anewhnaccount2 »

TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

It is happening: https://github.com/faster-cpython/ideas

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#18

Post by adgjlsfhk1 »

TekMol wrote: post #14

Python itself has to become faster. For the love of god. Make it as fast as PHP. My biggest pain point in computer science is that for every project I have to decide to either cope with PHP's butt-ugly namespace system or with Python's masochistic slowness. Please, Python developers. Take a closer look at how PHP achieves its speed, hold all other development and copy whatever they do! The result would be heaven.

would python being 3x faster make a difference? you'd still have so call out to an actually fast language when you want your code to be fast.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#19

Post by jakobnissen »

jkrubin wrote: post #11
jakobnissen wrote: post #10

Earlier quoted context omitted.

Don't be unreasonable. Numerical Julia code is just as fast, precisely because it's easy to design efficient high-level APIs and let the compiler optimise it. Try timing Julia's IO, or it's string processing, or it's hash tables, or its array allocation or it's GC - you know, nearly everything other that mathematical operations on floats or ints. Nearly everything leaves performance on the table. Sure Julia is fast,…

What’s your point?

The point is that when adgjlsfhk1 doubts that Julia is slower than C, he does so from a background where "data" is a synonym for arrays of numbers, and "computing" means processing these arrays. Julia shines here, precisely because the people developing Julia have a similar focus.

In contrast, if your performance bottlenecks are IO, parsing and hash table operations, then Julia's performance will be in par with Python and get absolutely crushed by Rust (I have less experience with C, but would imagine Rust and C are about equally fast).

The major difference with Python is that Julia's implementation of these things _could_ be fast in a way that Python's can't. They just aren't.

Re: JNumPy: Writing high-performance C extensions for Python in minutes

#20

Post by adgjlsfhk1 »

jakobnissen wrote: post #19
jkrubin wrote: post #11

Earlier quoted context omitted.

What’s your point?

The point is that when adgjlsfhk1 doubts that Julia is slower than C, he does so from a background where "data" is a synonym for arrays of numbers, and "computing" means processing these arrays. Julia shines here, precisely because the people developing Julia have a similar focus. In contrast, if your performance bottlenecks are IO, parsing and hash table operations, then Julia's performance will be in par with Pytho…

on par with python is way off. Julia in these areas definitely isn't fully optimized, but the performance will be much closer to something like Java for these cases. a dict in Julia will still have 1 pointer indirect compared to the python one where all your objects have 16 bytes overhead and an extra pointer indirection.
Post reply on HN