Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cython is 20

blog.behnel.de

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Cython is 20

#61

Post by jcelerier »

erwincoumans wrote: post #28

I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++. https://github.com/wjakob/nanobind If you really care about performance called from Python, consider something like NVIDIA Warp (Preview). Warp jits and runs your code on CUDA or CPU. Although Warp targets physics simulation, geometry processing, and proced…

if the object you wanna bind fits into the mold of "an algorithm with inputs and outputs, and some helper methods" I've got automatic binding of a limited set of C++ features working in https://github.com/celtera/avendish ; for now I've been using pybind11 but I guess everything I need is supported by nanobind so maybe i'll do the port...

Re: Cython is 20

#62

Post by victoryhb »

I tried to learn Cython last year, but was thwarted by two issues: (1) its syntax was too ugly for my taste and support for the pure Python mode was immature; (2) performance bottlenecks were opaque and hard to profile (at least for beginners). I ended up picking up Nim, a language with Python-like syntax and C-like performance, and was productive within hours (literally). I never looked back.

Re: Cython is 20

#63

Post by erwincoumans »

logicchains wrote: post #60
erwincoumans wrote: post #28

I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++. https://github.com/wjakob/nanobind If you really care about performance called from Python, consider something like NVIDIA Warp (Preview). Warp jits and runs your code on CUDA or CPU. Although Warp targets physics simulation, geometry processing, and proced…

>I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++ Why would you recommend that? It's all way more effort than just writing Cython, especially in a Jupyter Notebook. And Cython code can be just as fast as C/C++ code unless you're doing something really fancy. It's a bunch of work for no benefit. >Warp jits…

Warp generates C/C++ code, that can be trivially used in a pure C++ or CUDA project without issues. So it is not strictly jit, since it calls the regular ahead-of-time compiler (gcc, llvm or nvcc) only when de code changes (using hashes to check for changes), so performance is good. Also, random non-vectorizable branchy code will run fine on cpu with Warp, but you loose many benefits indeed.

Agreed, if you have bad performing spaghetti Python code, none of those tools are going to help indeed. Then I would rather rewrite it all in C/C++ instead of fiddling with Cython.

Re: Cython is 20

#64

Post by wjakob »

logicchains wrote: post #60
erwincoumans wrote: post #28

I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++. https://github.com/wjakob/nanobind If you really care about performance called from Python, consider something like NVIDIA Warp (Preview). Warp jits and runs your code on CUDA or CPU. Although Warp targets physics simulation, geometry processing, and proced…

>I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++ Why would you recommend that? It's all way more effort than just writing Cython, especially in a Jupyter Notebook. And Cython code can be just as fast as C/C++ code unless you're doing something really fancy. It's a bunch of work for no benefit. >Warp jits…

> Why would you recommend that? [..] It's a bunch of work for no benefit.

nanobind/pybind11 (co-)author here. The space of python bindings is extremely diverse and on the whole probably looks very different from your use case. nanobind/pybind11 target the 'really fancy' case you mention specifically for codebases that are "at home" in C++, but which want natural Pythonic bindings. There is near-zero overlap with Cython.

Re: Cython is 20

#65

Post by erwincoumans »

wjakob wrote: post #64
logicchains wrote: post #60

Earlier quoted context omitted.

>I would recommend considering using NanoBind, the follow up of PyBind11 by the same author (Wensel Jakob), and move as much performance critical code to C or C++ Why would you recommend that? It's all way more effort than just writing Cython, especially in a Jupyter Notebook. And Cython code can be just as fast as C/C++ code unless you're doing something really fancy. It's a bunch of work for no benefit. >Warp jits…

> Why would you recommend that? [..] It's a bunch of work for no benefit. nanobind/pybind11 (co-)author here. The space of python bindings is extremely diverse and on the whole probably looks very different from your use case. nanobind/pybind11 target the 'really fancy' case you mention specifically for codebases that are "at home" in C++, but which want natural Pythonic bindings. There is near-zero overlap with Cyth…

Yes, I assumed everyone who cares about performance (or who writes large programs) is also a C++ or CUDA programmer. Don't tell me that is not the case :-)

Re: Cython is 20

#66

Post by marky1991 »

coliveira wrote: post #58
dagw wrote: post #41

Earlier quoted context omitted.

People have. See PyPy and Pythran for two examples currently under active development. Instagram has such a project as well that they recently released. I know there have been others. None of them seem to catch on. It seems that most people don't actually want a faster subset of python. They want either all of python or none of python (by switching to another language all together)

Pypy is not really practical. It requires you to stop using many features of Python. What Python needs is a fast subset that is supported within the main implementation.

What do you mean? Pypy supports all of python. (not necessarily all of the C API, but that's somewhat a WIP and somewhat a result of the overly-expansive C API.)

Re: Cython is 20

#67

Post by tristan957 »

jokoon wrote: post #19
physicsguy wrote: post #2

I love Cython. I really feel like it's the right balance of usability and allowing you to do what you want/need. Want to make your code a bit faster? Write Python with type annotations. Want to call a C library? Just import the header, and then use it from a function. Pybind11 is also great, but quite different in aims - I feel like it's more like a project for C++ programmers wanting to expose functionality to Pytho…

Any benchmarks for type annotations? I already wrote a few patch for pysfml, which is written in cython, it was a bit awkward, and now I'm asking myself if cython is really the right tool to write bindings, compared to cpython, for example.

For what it's worth I wrote Python bindings using Cython for our open source C-API storage engine and the performance was fairly close to on par with C.

Re: Cython is 20

#68

Post by poulpy123 »

linspace wrote: post #12
fname11 wrote: post #11

Earlier quoted context omitted.

This is not going to happen. GvR has successfully ignored Cython and PyPy for decades and has attached himself to a JIT project at Microsoft (has anything emerged?). CPython is in the hands of not really productive bigcorp representatives who care about large legacy code bases. My guess is that CPython will be largely the same in 10 years, with the usual widely hyped initiatives that go nowhere ("need for speed etc."…

> who care about large legacy code bases It's clear that Python's main strength is its vast libraries, priority number one is not breaking them. If it could be possible to speed up Python without breaking changes I would be surprised precisely because with so much large codebases speed and efficiency would translate directly to money.

They really missed an opportunity when they made the switch from py2 to 3 to break things a bit more but give more improvement in exchange

Re: Cython is 20

#69

Post by travisoliphant »

coliveira wrote: post #40
dagw wrote: post #38

Earlier quoted context omitted.

Writing a much faster language runtime for a language that looks quite a bit like Python is easy. The hard problem is writing a faster language runtime that is 100% compatible with all current python programs (and their extensions) out in the world.

But this is not necessary. The python developers just need to specify this fast subset of the language, and let people use to create libraries. Over time, we would have a growing set of libraries written in the fast subset.

This is related to the idea of EPython that we are working on (as we have funding): https://github.com/epython-dev/epython

It currently emits Cython for the C-backend (and PyIodide). It is very alpha currently, but if people are interested in helping, get in touch.

Re: Cython is 20

#70

Post by ergo14 »

DaedPsyker wrote: post #15
ergo14 wrote: post #4

Earlier quoted context omitted.

We speed up our ML code 40 times using it.

Would that be in the data loading that you are getting the most benefit? I'm curious, since most of the big libraries are already just cuda calls anyway but I'm always interested in anything to speed up the full process.

In this case was multicore computation without GIL if i remember correctly.
Post reply on HN