Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A tail calling interpreter for Python (already landed in CPython)

blog.reverberate.org

Post reply on HN
Print view View original

71–80 of 93 posts

Re: A tail calling interpreter for Python (already landed in CPython)

#71

Post by Qem »

pinoy420 wrote: post #34
pansa2 wrote: post #24

Earlier quoted context omitted.

Any examples? The biggest change since Guido stepped down has been the addition of pattern matching, which he was strongly in favour of. Moreover, Guido is in favour of ongoing addition of major new features (like pattern matching), worrying that without them Python would become a “legacy language”: https://discuss.python.org/t/pep-8012-frequently-asked-quest...

Pattern matching seems like a cool feature that was added just because it was cool. I think the syntax is really odd too - apparently to “be pythonic”. I really see no use for it other than to “look smart”. The fact that case match (switch case is a much better description) is expanded to practically a huge if else is disturbing. Similarly the walrus operator. Other than an answer to “what is a new feature of python…

> Similarly the walrus operator. Other than an answer to “what is a new feature of python that you like” interview trivia question, really, who has actually used it?

At least in my case I use it all the time, to avoid duplicated operations inside comprehensions.

Re: A tail calling interpreter for Python (already landed in CPython)

#72

Post by cempaka »

pjmlp wrote: post #55
flakes wrote: post #25

Earlier quoted context omitted.

> Python has been losing users to Rust. I don't entirely understand that, other than everyone saying how Rust's tooling is so much better. Not to rust, but to Go and C++ for myself. The biggest motivating factor is deployment ease. It is so difficult to offer a nice client install process when large virtual environments are involved. Static executables solve so many painpoints for me in this arena. Rust would probabl…

As someone that always kept a foot on C++ land, dispite mostly working on managed languages, I would that by C++17 (moreso now in C++23), dispite all its quirks and warts, C++ has become good enough that I can write Python like code with it. Maybe it is only a thing to those of us already damaged with C++, and with enough years experience using it, but there are still plenty of such folks around to matter, specially…

Are there any books or curricula you'd recommend to someone starting out, who wants to learn a more modern style? My main worry is just that everything is going to be geared to C++11 (or worse, 98).

Re: A tail calling interpreter for Python (already landed in CPython)

#73

Post by pjmlp »

cempaka wrote: post #72
pjmlp wrote: post #55

Earlier quoted context omitted.

As someone that always kept a foot on C++ land, dispite mostly working on managed languages, I would that by C++17 (moreso now in C++23), dispite all its quirks and warts, C++ has become good enough that I can write Python like code with it. Maybe it is only a thing to those of us already damaged with C++, and with enough years experience using it, but there are still plenty of such folks around to matter, specially…

Are there any books or curricula you'd recommend to someone starting out, who wants to learn a more modern style? My main worry is just that everything is going to be geared to C++11 (or worse, 98).

Yes, from Bjarne Stroustroup himself,

A Tour of C++, preferably the 2nd edition

Programming -- Principles and Practice Using C++, preferably the 3rd edition

Re: A tail calling interpreter for Python (already landed in CPython)

#75

Post by IgorPartola »

sieve wrote: post #27
IgorPartola wrote: post #11

Earlier quoted context omitted.

Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. I do think it can probably hit pause on adding more syntax but at least everything it adds is backwards compatible. You won’t be writing a 3D FPS game engine in Python but you definitely can do a whole lot of real time data processing, batch processing, scientific computing, web and native applications, etc. before…

> If your only metric for a language is speed then nothing really beats hand crafted assembly Only if you know the micro-architecture of the processor you are running on at great depth and can schedule the instructions accordingly. Modern compilers and vms can do crazy stuff at this level. > Python is fast enough for a whole set of problems AND it is a pretty, easy to read and write language. It is definitely easy to…

Will your program ever be fast if you don’t learn the microarchitecture of your CPU first? :)

PyPy is a valid option and one I would explore if it fits what you are doing.

Re: A tail calling interpreter for Python (already landed in CPython)

#76

Post by cempaka »

pjmlp wrote: post #73
cempaka wrote: post #72

Earlier quoted context omitted.

Are there any books or curricula you'd recommend to someone starting out, who wants to learn a more modern style? My main worry is just that everything is going to be geared to C++11 (or worse, 98).

Yes, from Bjarne Stroustroup himself, A Tour of C++, preferably the 2nd edition Programming -- Principles and Practice Using C++, preferably the 3rd edition

Thanks!

Re: A tail calling interpreter for Python (already landed in CPython)

#77

Post by mversiotech »

pjmlp wrote: post #73
cempaka wrote: post #72

Earlier quoted context omitted.

Are there any books or curricula you'd recommend to someone starting out, who wants to learn a more modern style? My main worry is just that everything is going to be geared to C++11 (or worse, 98).

Yes, from Bjarne Stroustroup himself, A Tour of C++, preferably the 2nd edition Programming -- Principles and Practice Using C++, preferably the 3rd edition

The latest edition of "A Tour of C++" is the 3rd one, from 2022. Is there any specific reason why you would recommend the 2nd edition (from 2018) over that one?

Re: A tail calling interpreter for Python (already landed in CPython)

#78

Post by dragonwriter »

riffraff wrote: post #37
pansa2 wrote: post #24

Earlier quoted context omitted.

Any examples? The biggest change since Guido stepped down has been the addition of pattern matching, which he was strongly in favour of. Moreover, Guido is in favour of ongoing addition of major new features (like pattern matching), worrying that without them Python would become a “legacy language”: https://discuss.python.org/t/pep-8012-frequently-asked-quest...

I was thinking of the walrus operator, various f-string changes, relenting on the "GIL removal must not cost performance" stance (although"covered" by other improvements), things like that. I don't follow python closely so it may 100% be stuff that GvR endorsed too, or I'm mixing up the timelines. It just feels to me that python is changing much faster than it did in the 2.x days.

> python is changing much faster than it did in the 2.x days.

I think part of the reason Guido stepped down was that the BDFL structure created too much load on him dealing with actual and potential change, so its unsurprising that the rate of change increased when the governance structure changed to one that managed change without imposing the same load on a particular individual.

Re: A tail calling interpreter for Python (already landed in CPython)

#79

Post by dragonwriter »

pinoy420 wrote: post #34
pansa2 wrote: post #24

Earlier quoted context omitted.

Any examples? The biggest change since Guido stepped down has been the addition of pattern matching, which he was strongly in favour of. Moreover, Guido is in favour of ongoing addition of major new features (like pattern matching), worrying that without them Python would become a “legacy language”: https://discuss.python.org/t/pep-8012-frequently-asked-quest...

Pattern matching seems like a cool feature that was added just because it was cool. I think the syntax is really odd too - apparently to “be pythonic”. I really see no use for it other than to “look smart”. The fact that case match (switch case is a much better description) is expanded to practically a huge if else is disturbing. Similarly the walrus operator. Other than an answer to “what is a new feature of python…

I don't use pattern matching much, but I use walrus fairly regularly.

Re: A tail calling interpreter for Python (already landed in CPython)

#80

Post by haberman »

riffraff wrote: post #28

How does this differ from direct threading interpreters? It seems like it solves the same problem (saving the function call overhead) and has the same downsides (requires non-standard compiler extensions) EDIT: it seems the answer is that compilers do not play well with direct-threaded interpreters and they are able to perform more/better optimizations when looking at normal-sized functions rather than massive blocks…

This is a great summary. When Mike wrote the message you linked, his conclusion was that you have to drop to assembly to get reasonable code for VM interpreters. Later we developed the "musttail" technique which was able to match his assembly language sequences using C. This makes C a viable option for VM interpreters, even if you want best performance, as long as your compiler supports musttail.

> they are able to perform more/better optimizations when looking at normal-sized functions rather than massive blocks

It's not the size of the function that is the primary problem, it is the fully connected control flow that gums everything up. The register allocator is trying to dynamically allocate registers through each opcode's implementation, but it also has to connect the end of every opcode with the beginning of every opcode, from a register allocation perspective.

The compiler doesn't understand that every opcode has basically the same set of "hot" variables, which means we benefit from keeping those hot variables in a fixed set of registers basically all of the time.

With tail calls, we can communicate a fixed register allocation to the compiler through the use of function arguments, which are always passed in registers. When we pass this hot data in function arguments, we force the compiler to respect this fixed register allocation, at least at the beginning and the end of each opcode. Given that constraint, the compiler will usually do a pretty good job of maintaining that register allocation through the entire function.

Post reply on HN