Live data from Hacker News

Nuitka: An extremely compatible Python compiler

nuitka.net

41–50 of 88 posts

Re: Nuitka: An extremely compatible Python compiler

#41

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine? Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Just in general why should I be using Nuitka?

> Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Mind sharing that code? I'm sure someone would like to be able to look into this.

I don't have it anymore, but it was something stupid like:

    import time
    start = time.time()
    list = []
    for i in range(10000000):
        list.append(i)
    sum = 0
    for item in list:
        sum = sum + i
    print(time.time() - start)
I am not superduper python expert, but I know that arrays/lists are the stupidest thing you can use so that's why I did stupid things with them. So before someone posts more optimized version the code was suppose to be shit since I assumed that the compiler would optimize it.

I expected the binary to essentially be

    1. start = time.time()
    2. pre-populated list since it should be constant e.g. list = [1,2,3,...] or completely remove it since it is not used anywhere
    3. pre-filled sum since it too is constant or same as above, removed since it is not used anywhere
    4. getting second time.time() and printting the difference
But there was no compilation time optimization. Granted this wasn't promised, but I just assumed it was in there since it is compiling.

Re: Nuitka: An extremely compatible Python compiler

#42

How is this more compatible than just python? Since it requires libpython so you anyway need to have python installed on the target machine? Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec). Just in general why should I be using Nuitka?

nuitka takes your python code and all its deps, turns it to c code, and compiles that into a standalone executable that is both faster and doesn't need a separate python vm to be installed.

Expect it does require libpython and at least on Ubuntu 20.04 it links against the dev version of the library. So not only can you not run the resulting binary on any random Ubuntu 20.04 you also need to install same version of the python and the "-dev" version.

I did try this. With the "stock" python3 (3.8.something) I couldn't get nuitka to compile my code (or the example from the docs). With python3.9 I could start the compilation, but I didn't have libpython-dev shared object and finally with python3.9-dev I was able to make the binary. However after transfering it to another Ubuntu 20.04 machine I couldn't run it since it was linked against libpython3.9-dev.

Just as side note. The same python code (i.e. the .py file) would run on both machines out of the box.

Re: Nuitka: An extremely compatible Python compiler

#43
post #35

Earlier quoted context omitted.

What sort of web applications do you work on where Python interpreter speed is the limiting factor? Usually, those applications are constrained by network throughput and context switches and page faults.

This argument is so tiring. That's part of why websites are so slow despite having insane hardware at their disposal. At my company, for a web app we run in production, we strive to get every response out of our infrastructure in under a millisecond, everything above that except for a few select endpoints is considered as a bug. By using sensible technology choices, it's not even that hard to do. A RDMS like postgres…

Right, I agree. My question is “what application is constrained by interpreter overhead.” It should be faster than milliseconds for almost any code, I would think.

I am agreeing with you, I don’t understand your response.

Re: Nuitka: An extremely compatible Python compiler

#44

It would be neat if the big companies like Dropbox, Instagram, Google, Oracle, Shopify, Stripe, whatnot building better Python/JavaScript/Ruby implementations would start building program analysis libraries for Python/JavaScript/Ruby so that more implementations could get this for free. For example, the homepage of Nuitka says they only just added support for constant folding and propagation. That's such low hanging…

An issue, though not an unsolvable one, is that you probably need a standardized representation of the AST to write these optimization passes against; it is not uncommon for compiler writers to disagree on what representation is the best for what purpose.

Then there is the question of optimizations that are (typically) easier/possible only at the code-generation phase.

All of this is solvable of course, but there needs to be some will to do so (or in the case of companies, enough commercial benefit).

Re: Nuitka: An extremely compatible Python compiler

#45

This project is super impressive. It makes me wonder why there's not a similarly mature JavaScript AOT compiler implementation for use in game development or other places where you want performance better than a VM and you're not allowed to JIT. A small recommendation for these release pages would be to link to the tag in whatever source host you're using! I just want to see the code.

Because it will require tons of resources to make it more performant that the current JS JITs and it will probably fail. So what you see is some games just embedding V8.

Re: Nuitka: An extremely compatible Python compiler

#46

This project is super impressive. It makes me wonder why there's not a similarly mature JavaScript AOT compiler implementation for use in game development or other places where you want performance better than a VM and you're not allowed to JIT. A small recommendation for these release pages would be to link to the tag in whatever source host you're using! I just want to see the code.

You'd be stuck with float numerics everywhere because there's no fallback to recompile if an attempt to use integer fails.

Re: Nuitka: An extremely compatible Python compiler

#47

Earlier quoted context omitted.

nuitka takes your python code and all its deps, turns it to c code, and compiles that into a standalone executable that is both faster and doesn't need a separate python vm to be installed.

Expect it does require libpython and at least on Ubuntu 20.04 it links against the dev version of the library. So not only can you not run the resulting binary on any random Ubuntu 20.04 you also need to install same version of the python and the "-dev" version. I did try this. With the "stock" python3 (3.8.something) I couldn't get nuitka to compile my code (or the example from the docs). With python3.9 I could star…

it has an option to build with libpython. so your comment is incorrect.

Re: Nuitka: An extremely compatible Python compiler

#48
post #35

Earlier quoted context omitted.

This argument is so tiring. That's part of why websites are so slow despite having insane hardware at their disposal. At my company, for a web app we run in production, we strive to get every response out of our infrastructure in under a millisecond, everything above that except for a few select endpoints is considered as a bug. By using sensible technology choices, it's not even that hard to do. A RDMS like postgres…

Are you guys hiring? xD Just to expand on your point: Sure you can let a chunk of code take a few seconds longer than it could with little overall effect. But stack a hundred of these up and all those seconds compound and suddenly become immensely important. A suit of armour with a million chinks isn't very effective armour. (As an aside, depending on your use case, it can be worth optimizing the big bottlenecks, hig…

> A suit of armour with a million chinks isn't very effective armour.

Counterpoint: https://en.wikipedia.org/wiki/Chain_mail

Re: Nuitka: An extremely compatible Python compiler

#49
post #8

Wow, this got my attention. I use Python for web application development. A 2-3x speedup would be very interesting for higher-load deployments.

If you are interested in speed you might get more from Cython (not to mistake it with CPython) or mypyc. The catch is that you need to specify types (in Cython you otherwise won't get speed gain, and mypyc will refuse to compile).

I remember reading an article somebody was describing that instead of even adopting project to work with Cython he/she outright started their project in Cython and found it beneficial.

Post reply on HN