Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tail calling interpreter for Python (already landed in CPython)

blog.reverberate.org

Post reply on HN
Print view View original

11–20 of 93 posts

Re: A tail calling interpreter for Python (already landed in CPython)

#11

Post by IgorPartola »

VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before you need to start considering a faster interpreter.

If your only metric for a language is speed then nothing really beats hand crafted assembly. All this memory safety at runtime is just overhead. If you also consider language ergonomics, Python suddenly is not a bad choice at all.

Re: A tail calling interpreter for Python (already landed in CPython)

#12

Post by saidinesh5 »

Recent discussion: https://news.ycombinator.com/item?id=42999672

Do check out the articles in the top most comment.. about how tail call optimization gets you faster interpreters.

It completely eliminates the overhead of function calls in the generated machine code while you still your code modularly using functions.

Re: A tail calling interpreter for Python (already landed in CPython)

#13

Post by dragonwriter »

rpcope1 wrote: post #3
thunkingdeep wrote: post #2

This does NOT mean Python will get Tail Call Optimization, as Guido cannot be shown The Light, and has decided.

That's probably one of the more frustrating things about Python. Each release it gets all sorts of questionable new syntax (including the very strange pattern matching "feature" that kind of sucks compared to something like Erlang or Scala), but we never get real useful quality of life improvements for basic functional programming like TCO or multi line lambdas

> we never get real useful quality of life improvements for basic functional programming like TCO or multi line lambdas

A lambda can be as big of an expression as you want, including spanning multiple lines; it can't (because it is an expression) include statements, which is only different than lambdas in most functional languages in that Python actually has statements.

Re: A tail calling interpreter for Python (already landed in CPython)

#14

Post by VWWHFSfQ »

IgorPartola wrote: post #11
VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before…

I guess I'm wondering what is the point of tail-call optimizations, or even async/await when it's all super slow and bounded by the runtime itself? There are basically no improvements whatsoever to the core cpython runtime. So really what is all this for? Some theoretical future version of Python that can actually use these features in an optimal way?

Re: A tail calling interpreter for Python (already landed in CPython)

#15

Post by maxwelljoslyn »

VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

VWWHFSfQ, you may already know this, but: I recommend this talk by Armin Ronacher (Flask creator) on how Python's implementation internals contribute to the difficulties of making Python faster.

https://www.youtube.com/watchv=qCGofLIzX6g

One case study Ronacher gets into is the torturous path taken through the Python interpreter (runtime?) when you evaluate `__add__`. Fascinating stuff.

Re: A tail calling interpreter for Python (already landed in CPython)

#16

Post by mattbillenstein »

VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

The JIT will improve - you can also use PyPy to get speedups on programs that don't use a ton of C extensions.

Also, free-threading is coming so we'll have threads soon.

I don't know if Python can every really be fast as by design, objects are scattered all over memoryand even things like iterating a list, you're chasing pointers to PyObject all over the place - it's just not cache friendly.

Re: A tail calling interpreter for Python (already landed in CPython)

#17

Post by tcoff91 »

VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

I think if you want python but fast then Mojo is your only hope.

EDIT: yes and there’s pypy as well as pointed out below. Basically you gotta use an alternative python implementation of some kind.

Re: A tail calling interpreter for Python (already landed in CPython)

#18

Post by pansa2 »

tcoff91 wrote: post #17
VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

I think if you want python but fast then Mojo is your only hope. EDIT: yes and there’s pypy as well as pointed out below. Basically you gotta use an alternative python implementation of some kind.

There’s always PyPy - it’s much faster than CPython and, unlike Mojo, is ready to use today.

Re: A tail calling interpreter for Python (already landed in CPython)

#19

Post by beagle3 »

thunkingdeep wrote: post #2

This does NOT mean Python will get Tail Call Optimization, as Guido cannot be shown The Light, and has decided.

It is not an optimization ; it changes program semantics - converts programs that will run out of stack eventually regardless of the amount of available memory (and raise exceptions an the process, for example, which a program might rely on. Either way, semantics are changed)

It should only be called Tail Call Elimination.

Re: A tail calling interpreter for Python (already landed in CPython)

#20

Post by haberman »

saidinesh5 wrote: post #12

Recent discussion: https://news.ycombinator.com/item?id=42999672 Do check out the articles in the top most comment.. about how tail call optimization gets you faster interpreters. It completely eliminates the overhead of function calls in the generated machine code while you still your code modularly using functions.

Yes, that is the same article linked in the first sentence of this "update" article. :)

I published this technique four years ago, and it's very exciting to see that others have taken up the cause and done the work to land it in CPython.

Post reply on HN