Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tail calling interpreter for Python (already landed in CPython)

blog.reverberate.org

Post reply on HN
Print view View original

41–50 of 93 posts

Re: A tail calling interpreter for Python (already landed in CPython)

#41

Post by throwaway81523 »

VWWHFSfQ wrote: post #14
IgorPartola wrote: post #11

Earlier quoted context omitted.

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before…

I guess I'm wondering what is the point of tail-call optimizations, or even async/await when it's all super slow and bounded by the runtime itself? There are basically no improvements whatsoever to the core cpython runtime. So really what is all this for? Some theoretical future version of Python that can actually use these features in an optimal way?

This TCO is in how the CPython interpreter works, not in making Python itself tail recursive. Some of the C code in the interpreter has been reorganized to put some calls into tail position where the C compiler turns them into jumps. That avoids some call/return overhead and makes the interpreter run a little faster. It's still interpreting the same language with the same semantics.

Re: A tail calling interpreter for Python (already landed in CPython)

#42

Post by lstodd »

podunkPDX wrote: post #23
IgorPartola wrote: post #11

Earlier quoted context omitted.

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before…

> You won’t be writing a 3D FPS game engine in Python While Eve Online isn’t an FPS, it is an MMORPG written in stackless Python, and seems to be doing OK.

It was, once.

Nowadays (for about 12 years already I think) there is nothing much stackless about it.

The concept was nice. Stackless and greenlets.. yess. But the way they rewrote C stack just killed caches. Even a naive reimplementation just using separate mmapped stacks and wrapping the whole coro concept under then-Python's threading module instantly gained like 100x speedup on context switch loads like serving small stuff over HTTP.

Edit: Though at this point it didn't much differ from ye olde FreeBSD N:M pthread implementation. Which ended badly if anyone can remember.

Re: A tail calling interpreter for Python (already landed in CPython)

#43

Post by beagle3 »

dragonwriter wrote: post #26
beagle3 wrote: post #19

Earlier quoted context omitted.

It is not an optimization ; it changes program semantics - converts programs that will run out of stack eventually regardless of the amount of available memory (and raise exceptions an the process, for example, which a program might rely on. Either way, semantics are changed) It should only be called Tail Call Elimination.

By that standard, any optimization that changes scaling in any dimension changes semantics, which, well, I’m not saying its wrong, but I would say it is exactly what people looking for optimization want.

I disagree.

An optimization that speeds a program by x2 has the same effect as running on a faster CPU. An optimization that packs things tighter into memory has the same effect as using more memory.

Program semantics are usually referred to as “all output given all input, for any input configuration” but ignoring memory use or CPU time, provided they are both finite (but not limited).

TCE easily converts a program that will halt, regardless of available memory, to one that will never halt, regardless of available memory. That’s a big change in both theoretical and practical semantics.

I probably won’t argue that a change that reduces an O(n^5) space/time requirement to an O(1) requirement is a change in semantics, even though it practically is a huge change. But TCE changes a most basic property of a finite memory Turing machine (halts or not).

We don’t have infinite memory Turing machines.

edited: Turing machine -> finite memory Turing machine.

Re: A tail calling interpreter for Python (already landed in CPython)

#44

Post by vrighter »

jgalt212 wrote: post #5
rpcope1 wrote: post #3

Earlier quoted context omitted.

That's probably one of the more frustrating things about Python. Each release it gets all sorts of questionable new syntax (including the very strange pattern matching "feature" that kind of sucks compared to something like Erlang or Scala), but we never get real useful quality of life improvements for basic functional programming like TCO or multi line lambdas

The utility value of multi-line lambdas is real, but the readability of these is terrible. And Python prizes readability. So you know where this initiative will end up.

I personally find python to be highly UNreadable, especially with all of its syntax and braindead scoping rules

Re: A tail calling interpreter for Python (already landed in CPython)

#45

Post by vrighter »

IgorPartola wrote: post #11
VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before…

everything it adds is by default backwards compatible, because old programs didn't use it, because it wasn't there yet, and so won't break.

Python's problem is that the non-new stuff is not always backwards compatible. It happens way too often that A new python version comes out and half the python programs on my system just stop working.

Re: A tail calling interpreter for Python (already landed in CPython)

#47

Post by hathawsh »

pinoy420 wrote: post #36
tomsmeding wrote: post #35

Earlier quoted context omitted.

Modern C compilers are able to transform something like this: for (int i = 0; i To: a += n * (n+1) / 2; Is this an optimisation or a change in program semantics? I've never heard anyone call it anything slse than an optimisation.

Amazing it can do that. How does it work? That definitely does seem to change its semantics to me. I am not a c expert but this surely has problems the previous one doesn’t?

It does change the semantics if n is negative or large enough to cause an overflow. The challenge for the compiler is to somehow prove that neither of those things can happen.

Re: A tail calling interpreter for Python (already landed in CPython)

#48

Post by pansa2 »

pinoy420 wrote: post #34
pansa2 wrote: post #24

Earlier quoted context omitted.

Any examples? The biggest change since Guido stepped down has been the addition of pattern matching, which he was strongly in favour of. Moreover, Guido is in favour of ongoing addition of major new features (like pattern matching), worrying that without them Python would become a “legacy language”: https://discuss.python.org/t/pep-8012-frequently-asked-quest...

Pattern matching seems like a cool feature that was added just because it was cool. I think the syntax is really odd too - apparently to “be pythonic”. I really see no use for it other than to “look smart”. The fact that case match (switch case is a much better description) is expanded to practically a huge if else is disturbing. Similarly the walrus operator. Other than an answer to “what is a new feature of python…

Yeah, it was added to tick the box for people who ask "does Python have pattern matching?"

If you look at the feature in detail, and especially how it clashes with the rest of the language, it's awful. For example:

https://x.com/brandon_rhodes/status/1360226108399099909

Re: A tail calling interpreter for Python (already landed in CPython)

#49

Post by cudder »

maxwelljoslyn wrote: post #15
VWWHFSfQ wrote: post #10

Will Python ever get fast? Or even _reasonably_ fast? The answer is no, it will not. Instead they'll just keep adding more and more syntax. And more and more ways to do the same old things. And they'll say that if you want "fast" then write a native module that we can import and use. So then what's the point? Is Python really just a glue language like all the rest?

VWWHFSfQ, you may already know this, but: I recommend this talk by Armin Ronacher (Flask creator) on how Python's implementation internals contribute to the difficulties of making Python faster. https://www.youtube.com/watchv=qCGofLIzX6g One case study Ronacher gets into is the torturous path taken through the Python interpreter (runtime?) when you evaluate `__add__`. Fascinating stuff.

Your link is broken, here's a working one: https://www.youtube.com/watch?v=qCGofLIzX6g

Re: A tail calling interpreter for Python (already landed in CPython)

#50

Post by sieve »

flakes wrote: post #25
throwaway81523 wrote: post #6

Earlier quoted context omitted.

Python has always been unashamedly imperative, with some functional features entering by slipping through the cracks. The pattern matching thing seemed ok to me when I tried it, but I haven't used it except briefly, since I'm still mostly on Python 3.9. Interestingly, Python has been losing users to Rust. I don't entirely understand that, other than everyone saying how Rust's tooling is so much better.

> Python has been losing users to Rust. I don't entirely understand that, other than everyone saying how Rust's tooling is so much better. Not to rust, but to Go and C++ for myself. The biggest motivating factor is deployment ease. It is so difficult to offer a nice client install process when large virtual environments are involved. Static executables solve so many painpoints for me in this arena. Rust would probabl…

> For tooling shipped to client environments, I now tend to steer clear of it.

A guy on r/WritingWithAI is building a new writing assistant tool using python and pyQt. He is not a SE by trade. Even so, the installation instructions are:

- Install Python from the Windows app store

- Windows + R -> cmd -> pip install ...

- Then run python main.py

This is fine for technical people. Not regular folks.

For most people, these incantations to be typed as-is in a black window mean nothing and it is a terrible way of delivering a piece of software to the end-user.

Post reply on HN