Nuitka: An extremely compatible Python compiler

I first found it back in 2015 when I worked at a company where we built a python based desktop application as a part of our industrial control system. Nuitka provided better performance then pyinstaller/cx_freeze while it was still simple to work with. Back then there were still a few incompatibilities but I've followed the project throughout the years and it's mature like a fine wine since.

Re: Nuitka: An extremely compatible Python compiler

#4

Post by m4r35n357 » Thu, Sep 02, 2021, 10:00 AM UTC

Unlike most Python "accelerators" this seems to work OOTB with no faff!

Re: Nuitka: An extremely compatible Python compiler

#5

Post by aarchi » Thu, Sep 02, 2021, 10:01 AM UTC

Nuitka looks like a traditional ahead-of-time compiler using SSA form[0] and is written in Python. I’d be interested in seeing performance comparisons with PyPy, which uses the second Futamura protection and is written in a dialect of Python.

[0]: https://nuitka.net/doc/developer-manual.html#ssa-form-for-nu...

Re: Nuitka: An extremely compatible Python compiler

#6

Post by nextlevelwizard » Thu, Sep 02, 2021, 10:35 AM UTC

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine?

Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec).

Just in general why should I be using Nuitka?

Re: Nuitka: An extremely compatible Python compiler

#7

Post by nightcracker » Thu, Sep 02, 2021, 10:46 AM UTC

nextlevelwizard wrote: post #6

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine? Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Just in general why should I be using Nuitka?

I think the idea behind Nuitka is to compile Python into an executable, rather than interpret a script using a fully generic interpreter, but without (or with minimal) limitations, falling back to libpython's implementation if necessary.

> In the future Nuitka will be able to use type inferencing based on whole program analysis. It will apply that information in order to perform as many calculations as possible in C, using C native types, without accessing libpython.

I believe the difference with PyPy is that PyPy tries to do this using just-in-time compilation and nuitka uses ahead-of-time compilation.

Re: Nuitka: An extremely compatible Python compiler

#8

Post by philpem » Thu, Sep 02, 2021, 10:53 AM UTC

Wow, this got my attention. I use Python for web application development. A 2-3x speedup would be very interesting for higher-load deployments.

Re: Nuitka: An extremely compatible Python compiler

#9

Post by lennoff » Thu, Sep 02, 2021, 10:56 AM UTC

Tangentially related question: could Nuitka target WebAssembly, given that according to their overview page, they translate Python into C? Usually when it comes to Python -> WebAssembly, the biggest problem is the lack of GC in the WebAssembly spec (as far as I understand), and I'm wondering if this would be an issue for Nuitka as well...

Re: Nuitka: An extremely compatible Python compiler

#10

Post by codeflo » Thu, Sep 02, 2021, 11:54 AM UTC

lennoff wrote: post #9

Tangentially related question: could Nuitka target WebAssembly, given that according to their overview page, they translate Python into C? Usually when it comes to Python -> WebAssembly, the biggest problem is the lack of GC in the WebAssembly spec (as far as I understand), and I'm wondering if this would be an issue for Nuitka as well...

Of course you can build a GC in WebAssembly. You might have to avoid the native stack and lose some performance that way, but that shouldn’t be too bad I think.

One problem is that this GC doesn’t interact with the browser’s GC in any way. So you have painful memory management interactions when (for example) a DOM event handler references a WebAssembly object which in turn holds a reference to a browser object, possibly with cycles (so simple reference counting isn’t enough).

You can fall back to manual memory management here, but that’s painful to use. To make this work seamlessly, you need a way to trace references across both heaps in one swoop. Last I checked, there was standardization work underway to enable that.