Python-based compiler achieves orders-of-magnitude speedups
1–10 of 193 posts
Re: Python-based compiler achieves orders-of-magnitude speedups
#2There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path).
You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented in a python like code.
You can use a really fast language like C, Rust, C++,... create python wrappers with cython, swig, Boost.python, cffi,... and use python like glue code.
Python is not a fast languages as others, but there are tricks to make fast programs.
Re: Python-based compiler achieves orders-of-magnitude speedups
#3So the differences:
https://docs.exaloop.io/codon/general/differences
So more limited types (integers) and more type checking and collections have to have one kind of thing in them.
There are other python compilers though, like https://github.com/Nuitka/Nuitka
I wonder really what the advantages/disadvantages of these are?
Re: Python-based compiler achieves orders-of-magnitude speedups
#4previously: https://news.ycombinator.com/item?id=33908576
Re: Python-based compiler achieves orders-of-magnitude speedups
#5I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python?
I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc?
Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset of that language with C-API compatibility?
Re: Python-based compiler achieves orders-of-magnitude speedups
#6I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers.
However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing you lots of your benefits of JITing.
You could, hypothetically, introduce an entirely new API, which allowed for faster dynamic recompiling, but then you'd need to get every package anyone cares about to switch to that.
Re: Python-based compiler achieves orders-of-magnitude speedups
#7I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
This is Pypy and Rpython!
Re: Python-based compiler achieves orders-of-magnitude speedups
#8There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…
numba seems to hit the sweet spot for most numerics for me. it can get a little annoying with type inference issues but overall it seems the most concise and least hassle for moving loops into optimized machine code.
Re: Python-based compiler achieves orders-of-magnitude speedups
#9I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
Django makes use of a whole lot of Python’s fancypants stuff. For this reason, for instance, mypy doesn’t do well on Django projects without a purpose-built plugin. But I still take your point.
Re: Python-based compiler achieves orders-of-magnitude speedups
#10I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
Most web stuff, but Django especially, relies heavily on dynamic Python magic to make cuter APIs.