Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Python Interpreter Written in Python (2016)

aosabook.org

Post reply on HN
Print view View original

11–20 of 42 posts

Re: A Python Interpreter Written in Python (2016)

#11

Post by chubot »

I found this code very useful. I've been spelunking in CPython and mostly I understand it, and can find my way around.

But ceval.c is another beast entirely, being full of macros and gotos, not to mention being 5000 lines long.

The interpreter loop starts here: https://github.com/python/cpython/blob/master/Python/ceval.c...

So I appreciate seeing the algorithm laid out in Python. In particular it clarifies that there are three separate stacks:

1. call stack

2. block stack for try/except, loops, etc.

3. value stack for evaluating expressions

It also clarifies how generators work, which IMO is very difficult to follow from the C source (i.e. without a design doc to go along).

I wrote about some of my recent work here: https://www.reddit.com/r/oilshell/comments/8b0n6z/opyreadmem...

BTW I have shell scripts running under triple interpretation: CPython, byterun, and OSH itself :) This is just an experiment toward writing my own VM, not for the final product. The release binary doesn't use byterun at all.

EDIT: There is also a companion bytecode compiler that I mention here: http://www.oilshell.org/blog/2018/03/27.html (but I'm not using it, I'm using the one that used to be in the Python 2 stdlib, which is entirely separate from the one used by the interpreter itself.)

Re: A Python Interpreter Written in Python (2016)

#12

Post by chrisseaton »

niroze wrote: post #5
0x7f800000 wrote: post #2

But can the interpreter interpret itself?

It isn't a LISP :P

You don't need a language to be a LISP in order for them to be able to interpret themselves. There are several Java interpreters which can interpret themselves, for example.

Re: A Python Interpreter Written in Python (2016)

#13

Post by masklinn »

asperous wrote: post #10

Just in case anyone doesn't know, there's actually a real, live project that does this: https://pypy.org/ (Technically the base interpreter is a subset/restricted version of Python however). The reason this is done (my understanding), is that a the python source code for the base interpreter can be fed into a JIT generator, which produces a Python interpreter that can perform JIT optimizations.

> Technically the base interpreter is a subset/restricted version of Python however

The base interpreter is written in a subset/restricted version of Python, but it's still Python. You can run Pypy as an interpreter on top of CPython or pypy.

And IIRC the restrictions mostly have to do with static type inference so you're mostly limited in how dynamic/weird the program gets the other big limits being magic methods being unavailable (for user-defined types) and most of the stdlib being off-limits.

So it's Python in a straightjacket, but for most interpreter implementations it's probably fine, as long as you're not using an overly interesting parsing strategy (hello pratt parsers) an interpreter tends not to use the more dynamic/odd corners of the language I think.

Re: A Python Interpreter Written in Python (2016)

#15

Post by ubernostrum »

chubot wrote: post #11

I found this code very useful. I've been spelunking in CPython and mostly I understand it, and can find my way around. But ceval.c is another beast entirely, being full of macros and gotos, not to mention being 5000 lines long. The interpreter loop starts here: https://github.com/python/cpython/blob/master/Python/ceval.c... So I appreciate seeing the algorithm laid out in Python. In particular it clarifies that there…

You may find this useful:

https://leanpub.com/insidethepythonvirtualmachine

I read through it, and the Python-interpreter-in-Python article, and all the documentation of the 'dis' module, while prepping the talk I'm giving at PyCon next month (which is on bytecode). They were all good resources.

Re: A Python Interpreter Written in Python (2016)

#16

Post by make3 »

codetrotter wrote: post #7
jwilk wrote: post #3

(2016), according to the HTTP headers: Last-Modified: Sat, 09 Jul 2016 12:15:59 GMT

I’m going to start having my server send Last-Modified: Mon, 20 Apr 3018 13:37:00 GMT Then we will see what people say when my stuff is posted :^) Then again my stuff probably wouldn’t get much attention to begin with. But if it did... People would have to admit that they are all living in the present whereas I was living in the future.

pay your registrar to return a WHOIS on Mars. https://news.nationalgeographic.com/2016/10/planets-maps-exp...

Re: A Python Interpreter Written in Python (2016)

#18

Post by 00ajcr »

chubot wrote: post #11

I found this code very useful. I've been spelunking in CPython and mostly I understand it, and can find my way around. But ceval.c is another beast entirely, being full of macros and gotos, not to mention being 5000 lines long. The interpreter loop starts here: https://github.com/python/cpython/blob/master/Python/ceval.c... So I appreciate seeing the algorithm laid out in Python. In particular it clarifies that there…

Philip Guo has an excellent set of video lectures on CPython internals, which includes an overview of parts of ceval.c: http://pgbovine.net/cpython-internals.htm

Re: A Python Interpreter Written in Python (2016)

#20

Post by chubot »

00ajcr wrote: post #18
chubot wrote: post #11

I found this code very useful. I've been spelunking in CPython and mostly I understand it, and can find my way around. But ceval.c is another beast entirely, being full of macros and gotos, not to mention being 5000 lines long. The interpreter loop starts here: https://github.com/python/cpython/blob/master/Python/ceval.c... So I appreciate seeing the algorithm laid out in Python. In particular it clarifies that there…

Philip Guo has an excellent set of video lectures on CPython internals, which includes an overview of parts of ceval.c: http://pgbovine.net/cpython-internals.htm

I've watched several of those, and they are good. Though for me, playing with code is a different type of learning than watching videos. I should go back and watch the one on ceval though.
Post reply on HN