Pyjion – A Python JIT Compiler

Pyjion – A JIT for Python based upon CoreCLR - https://news.ycombinator.com/item?id=10984514 - Jan 2016 (23 comments)

https://link.springer.com/book/10.1007/3-540-48737-9

Java does have a formal semantics, with a whole chapter on its soundness.

> in general, cpython is the spec

Well there we go - turns out the written document doesn't cover everything after all. A second ago we were at 'Every guaranteed behavior of python is described clearly and concretely in these documents.' Turns out not.

I'm not criticising Python as being exceptionally bad, but we can certainly do it much better.

No, there's a chapter on the soundness of its type system. The spec being sound and the type system being sound are very different things. If we consider typescript to be JS's type system, then JS's type system is unsound. If we consider cpython in isolation, under the definition you're using, cpython cannot be unsound, as it is untyped, QED.

If you're talking about whether the language's type system is sound, asking "These documents are just informal prose. Are they sound?" isn't even a well defined question.

> I'm not criticising Python as being exceptionally bad, but we can certainly do it much better.

You absolutely were when you said "But I wish Ruby and Python were this explicit and easy to understand!"

So if you change some of your Cython code, for it to be used at runtime you need to invoke the Cython build tools to rebuild the new version of your python native module.

I usually use Cython for a small core of compute heavy operations, and leave the rest of the project as pure python. That way I only need to rebuild Cython code if I change something inside that small core.

> Is it 1-to-1 with standard Python?

Not for the best speedups, no.

E.g. you might be able to get a modest speedup, say 50%, taking a pure python file, renaming it to *.pyx, and getting Cython to compile it. But that's not why I use Cython. I use it when I have compute-heavy code that I want to run at native speed (think matrix-vector product type stuff), by carefully rewriting in Cython, thinking carefully about memory allocation, data structures (prefer C arrays!) and performance, it is fairly achievable to get a 500x speedup.

Cython relies on you writing specialised Cython code that is quite close to C code -- strongly typed Cython variables work like statically typed C variables, not dynamically typed Python names. You end up with Cython code that cannot be executed as if it were normal Python code by a python interpreter.

But, Python code can usually not be executed very efficiently, whereas Cython can translate small loops of strongly-typed numeric code into small loops of strongly-typed C code, which can often compile to very small loops of native CPU instructions, which then run blazing fast.

Under the hood, Cython works by translating the not-quite-python code into C code that uses the python interpreter's C extension API. Then it compiles the C code into a python native module using a C compiler.

I completely agree. Cython can become quite attractive if your alternative is "write a python extension library by hand in C / C++". I first started using Cython after doing exactly that, writing my extension library in C, then realising that Cython might save a lot of work in generating the bindings and packaging/distribution -- it did, and it ran at exactly the same speed as my pure C library with hand crafted python bindings. After that I've been pretty excited about Cython.

If you've got a python program that needs to do a core of compute-heavy work, if you were to optimise this by writing a C / C++ library for python to use, the work would be: (i) think hard about how the library design will enable performance, (ii) implement that high performance library in C / C++ , (iii) figure out the interface so that python can call into the library, and (iv) figure out how to package and distribute the library so it can be used by python programs.

Cython doesn't really help with parts (i) designing for performance or (ii) writing that high performance code. But it helps a lot with parts (iii) and (iv), generating Python bindings and producing wheel archives that can be managed by existing python package management tooling.