Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python-based compiler achieves orders-of-magnitude speedups

news.mit.edu

Post reply on HN
Print view View original

21–30 of 193 posts

Re: Python-based compiler achieves orders-of-magnitude speedups

#21

Post by doix »

DeathArrow wrote: post #19
ar9av wrote: post #2

There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…

Can you use Django with those optimisations or are they good mainly for scientific computing?

At my old gig we ran Django and FastAPI with pypy. I don't remember there being too many issues. One thing is that pypy versions lag behind official python, so if you're using bleeding edge stuff, it won't be supported in pypy yet.

That was a couple of years ago at this point, and I've not been in the python ecosystem since then, but I can only imagine things are getting better in that regard rather than worse.

Re: Python-based compiler achieves orders-of-magnitude speedups

#22

Post by a2800276 »

DeathArrow wrote: post #15

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

It wouldn't waste time garbage collecting to focus on trash talking instead.

Re: Python-based compiler achieves orders-of-magnitude speedups

#23

Post by rpep »

DeathArrow wrote: post #19
ar9av wrote: post #2

There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…

Can you use Django with those optimisations or are they good mainly for scientific computing?

You can certainly use it, but whether you see any benefit is going to strongly depend on your workload. If you're doing significant calculations in the API then it might be considerably more performant, but if your API is primarily retrieving things from the database and transforming it to JSON then you're going to be limited mostly by the database latency and so I wouldn't expect major improvements.

Re: Python-based compiler achieves orders-of-magnitude speedups

#24

Post by spokeonawheel »

DeathArrow wrote: post #15

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

do people actually talk about the kardashians still? I thought that was like 3 years ago

Re: Python-based compiler achieves orders-of-magnitude speedups

#25

Post by jstx1 »

DeathArrow wrote: post #19
ar9av wrote: post #2

There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…

Can you use Django with those optimisations or are they good mainly for scientific computing?

Have a look at Cinder - https://github.com/facebookincubator/cinder - it's Meta's performance oriented fork of CPython that they use to run Instagram (which is a big Django app).

Re: Python-based compiler achieves orders-of-magnitude speedups

#27

Post by sigmoid10 »

spokeonawheel wrote: post #24
DeathArrow wrote: post #15

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

do people actually talk about the kardashians still? I thought that was like 3 years ago

The article states that this was around 2017-2018.

Re: Python-based compiler achieves orders-of-magnitude speedups

#28

Post by kaba0 »

CJefferson wrote: post #6
wageslave99 wrote: post #5

I have no idea about compilers, so bear with me with this question: Can't we have a faster compiler for a subset of Python? I mean AFAIK the hard part of Python is that the language allows dynamic overwriting of attributes (or something like that). Is that feature actually needed for projects like Django, FastAPI, numpy, etc? Maybe I'm wrong, but the main idea I'd like to ask is, can we make a compiler for a subset o…

The problem turns out to be that it's maintaining the C-API compatibility which is the main thing which makes it hard to make Python fast, not the other stuff -- Javascript has most of the nasty things Python does, and it's plenty fast on browsers. However, maintaining C-API compatibility means you need to set up lots of data structures exactly how the C API requires, and maintaining and updating those ends up losing…

That’s why I found GraalVM’s approach intriguing. They provide a high level language API where you can simply write a language interpreter, and it will be able to JIT/AOT compile it down to fast machine code. But the most interesting aspect is that the IR they convert languages to is basically universal, so something like LLVM bitcode can also use the exact same representations.

So you can interpret (and later AOT compile as well) LLVM bitcode and python, and this approach will allow cross-language optimizations as well, which were not available at all before. But feel free to add a bit of JS/Java, etc to your code as well!

Re: Python-based compiler achieves orders-of-magnitude speedups

#29

Post by smcl »

DeathArrow wrote: post #17

It's not the first time it was tried to compile Python: https://en.wikipedia.org/wiki/IronPython I hope this time we will see better results.

I'd say Nuitka or Cython are maybe the more common ones when talking about this. IronPython is/was interesting in that instead compiling to python bytecode or machine code it targetted the .NET CLR, and iirc I saw some kind of JIT going on when I was digging around (so some things ended up as machine code), but it's not really one of the usual "compiles python to machine code" implementations.

Re: Python-based compiler achieves orders-of-magnitude speedups

#30

Post by pungentcomment »

DeathArrow wrote: post #15

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

It's all garbage collector.
Post reply on HN