71–80 of 193 posts
Surprised there is no comparison to MyPyC. That said the availability of a "JIT" compiler in the style of Numba but with much broader Python feature support sounds great to me.
https://github.com/facebookincubator/cinder/
Disclaimer: I used to work on it.
I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
Can't we have a faster compiler for a subset of Python? Check out Pythran, that is exactly what they've done.
Earlier quoted context omitted.
I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots…
The difference is that the other languages FFI don't expose internals like CPython does. For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code. When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
Don’t get me wrong. I’m not passing a value judgement on the maintainers. But the reasons don’t feel technical to me.
nit: 'Python-based' would imply to me that it's written in Python, but it looks like it's mostly C++ & LLVM: https://github.com/exaloop/codon/tree/develop/codon
Earlier quoted context omitted.
Hardly a nitpick. It's key to the claim.
It's a little sad because PyPy literally is written in (a restricted subset of) Python, hence the name.
code here: https://github.com/avinassh/fast-sqlite3-inserts
my blog post: https://avi.im/blag/2021/fast-sqlite-inserts/
Earlier quoted context omitted.
And it is severely underrated. Even though performance gain is aevrate around 4x-20x. Used in production and memory usage is also about 1/6th of CPython. Can get 10x perfromance easily in many cases.
> Used in production and memory usage is also about 1/6th of CPython I thought it would have higher memory usage? (based only on reading)
nit: 'Python-based' would imply to me that it's written in Python, but it looks like it's mostly C++ & LLVM: https://github.com/exaloop/codon/tree/develop/codon
A fair implication, but they mean "Python-based" in that the language the compiler implements is based on Python.
Considering at least 2 people have gone to look at the source and then come here to comment, it would have been a net benefit for all involved. Plus, what does it say about the potential quality of your compiler if you can't even make correct English statements? This seems easier to get right than if( x = *p++ )
> Python — which is typically orders of magnitude slower than languages like C Not to nit-pick...this has been characterized by a team who tested and compared a large set of languages against a wide range of application code. The number is, if I remember correctly, about 78x slower. I don't think "orders" of magnitude is entirely fair. Yes, Python is slow. I have made the mistake of trying to use it for time-critical…
78x is roughly two orders of magnitude in typical physics parlance. If you take a more CSy stance and count powers of two, it would be six to seven orders of magnitude. Sounds entirely fair to me.
In other words, the acceleration isn't measured against raw C implementations (where the 78x factor I quoted would be relevant). It is measured against Python or PyPy.
How much faster does Codon make your Python code. The answer seems to be somewhere around the 5x to 10x range.
In that context, and in the context of actual applications rather than hand-picked tests (how much can we optimize a loop), "orders of magnitude" seems to be an exaggeration.
BTW, MIT does this kind of thing all the time with their press releases. They have a brand to support with outlandish claims about everything that comes out of there. Those with frequent exposure to this kind of press release are wise to this. I've seen it for decades. It's marketing.
For me, when someone says "orders of magnitude" it means "massive". I tend to say "10 times faster", "50 times faster" even "100 times faster". I probably start using "orders of magnitude" faster at 1000x or when I am trying to explicitly make an impression on a mathematically-challenged audience. "Orders of magnitude" sounds great to that crowd.
I have never, in 40 years in CS/Engineering, heard anyone use powers-of-two when they say "orders of magnitude". Doing so would open you to serious misinterpretation. Engineers might say something like "a factor of 2 to the n" or something like that.
Can we please not? Humanity wasted close to 50 years optimizing compilers for one garbage language. Wasted unimaginable efforts, money and developer hours... and all could've been avoided if the same people dedicated a fraction of those resources to language design. Same thing happened with Java. And now the existence of a well-developed compiler became an argument in its own right in favor of choosing a bad language…
Quite sad to know that the dynamic nature of Python is preventing the speedups in the first place. I really hope there'll be a built-in optimizing JIT compiler without the limitations of PyPy, Codon, Nuitka, Numba, etc.
JavaScript/lua are dynamic and they are fast. It's other choices (GIL) which cause problem rather than the nature of the dynamic language space