> Faster than the speed of C
One must note that this is impossible, unless you have chosen to handicap the C-implementations while benchmarking. Borderline unethical IMO to put forth such a claim.
Python-based compiler achieves orders-of-magnitude speedups
61–70 of 193 posts
Re: Python-based compiler achieves orders-of-magnitude speedups
#62Earlier quoted context omitted.
The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers. However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing…
I really try hard to understand this argument and I must be missing something and must be super stupid. Don’t languages like JavaScript have this and yet they can still do JIT and the base runtime is still in C++? Java itself has an official way to invoke C programs from Java applications and still has a JIT. And Java also has AOT compilers. Sure. Crossing that FFI boundary is going to be expensive. But there’s lots…
The difference is that the other languages FFI don't expose internals like CPython does.
For example, JNI only exposes handles and you need to convert an handle to a pointer, so the runtime knows for the time being that handle is special and being used by native code.
When it is only an opaque handle, lots of optimizations can happen and the native code won't see them.
Re: Python-based compiler achieves orders-of-magnitude speedups
#63I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…
> Can't we have a faster compiler for a subset of Python? That's exactly how PyPy works.
PyPy isn't really a subset of Python. From pypy.org:
PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9Re: Python-based compiler achieves orders-of-magnitude speedups
#64> Python — which is typically orders of magnitude slower than languages like C
Not to nit-pick...this has been characterized by a team who tested and compared a large set of languages against a wide range of application code. The number is, if I remember correctly, about 78x slower. I don't think "orders" of magnitude is entirely fair. Yes, Python is slow. I have made the mistake of trying to use it for time-critical embedded applications. Never again.
Aside from this admittedly pedantic observation, the first thing that crossed my mind with regards to this tool --which sounds fantastic-- is that you would have to trust the correctness and reliability of your code to this translation layer. Not sure how to think about this other than to keep a mental note of it if using this tool.
Re: Python-based compiler achieves orders-of-magnitude speedups
#65>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D
Kardashians are famous for... being famous.
That and wealth. There’s plenty of billionaires living privately, but many people find rich people who spend extravagantly interesting.
Re: Python-based compiler achieves orders-of-magnitude speedups
#66There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…
Can you use Django with those optimisations or are they good mainly for scientific computing?
I had little trouble switching a Django app years ago but the results were mixed. Some complicated views and reports saw a hefty win but most of the app was database-limited and optimized to the point that there was no meaningful difference, except that PyPy used more RAM.
Re: Python-based compiler achieves orders-of-magnitude speedups
#67Earlier quoted context omitted.
You can certainly use it, but whether you see any benefit is going to strongly depend on your workload. If you're doing significant calculations in the API then it might be considerably more performant, but if your API is primarily retrieving things from the database and transforming it to JSON then you're going to be limited mostly by the database latency and so I wouldn't expect major improvements.
And yet the same supposedly "io bound" workloads (like "parse request, fetch something from DB, return it as JSON") still have widely difference performance characteristics in some languages vs others, with 10x to 100x requests handled per second...
2x, yes. If you’re seeing 100x you’re comparing different things like creating and serializing complex objects versus simple types or using a JSON parser which loads an entire document into objects versus one which only retrieved specific values.
Re: Python-based compiler achieves orders-of-magnitude speedups
#68I did a similar demo project five years ago when I was learning compiler theory. And it was implemented by Python.
Re: Python-based compiler achieves orders-of-magnitude speedups
#69Earlier quoted context omitted.
> Can't we have a faster compiler for a subset of Python? That's exactly how PyPy works.
PyPy isn't really a subset of Python. From pypy.org: PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9
Yes - I think they meant RPython, which the PyPy team developed explicitly as the easy to optimize safer subset of Python.
Re: Python-based compiler achieves orders-of-magnitude speedups
#70> Python — which is typically orders of magnitude slower than languages like C Not to nit-pick...this has been characterized by a team who tested and compared a large set of languages against a wide range of application code. The number is, if I remember correctly, about 78x slower. I don't think "orders" of magnitude is entirely fair. Yes, Python is slow. I have made the mistake of trying to use it for time-critical…
78x is roughly two orders of magnitude in typical physics parlance. If you take a more CSy stance and count powers of two, it would be six to seven orders of magnitude. Sounds entirely fair to me.