Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SPy: An interpreter and compiler for a fast statically typed variant of Python

antocuni.eu

Post reply on HN
Print view View original

31–40 of 133 posts

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#31

Post by brabel »

psychoslave wrote: post #28
a-french-anon wrote: post #23

Earlier quoted context omitted.

Common Lisp doesn't use (expensive) CLOS dispatch in the core language, e.g. to add two numbers or find the right equality operator. That's one known pain point due to CLOS having been "bolted-on" rather than part of the language which makes the divide between internal (using typecase and similar) and external (generic functions) dispatch pretty ugly; and gave use the eql/equal/equalp/etc... hell. Thing is that you n…

What is CLOS in this context?

Common Lisp Object System. The language amazing version of what OOP can be.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#32

Post by gjvc »

pjmlp wrote: post #21
cardanome wrote: post #18

Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have insight into this?

Just like Smalltalk and SELF, also Lisp Machines and Interlisp-D. Usually comes down from a urban myth that Python is special and there was no other dynamic language before it came to be. The JIT research on those platforms is what gave us leading JIT capabilities on modern runtimes, OpenJDK HotSpot traces back to Smalltalk and StrongTalk, while V8 traces back to SELF. Especially in Smalltalk and SELF, you can change…

Great explanation. Five years ago I did the genealogical work to discover that StrongTalk begat HotSpot (by virtue of having some of the same authors) It was quite a joy to discover!

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#33

Post by brabel »

cardanome wrote: post #18

Common Lisp also allows you to redefine everything at runtime but doesn't suffer from the same performance issues that Python has, does it? Doe anyone have insight into this?

Common Lisp is not a runtime, it’s a specification. Implementations are free to compile everything to fast native code, or to interpret everything. Various available implementations do that and everything in between. That said , SBCL and the commercial implementations can be extremely fast, especially if you specify types on tight loops. SBCL comes with a disassembler that shows you right in the REPL the Assembly a function compiles to so you can even get close to C performance.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#34

Post by AmazingTurtle »

ForceBru wrote: post #26
AmazingTurtle wrote: post #24

> SPy is [...] a compiler > SPy is not a "compiler for Python" I think it's funny how it's confusing from the first paragraph

Reading the next sentence clears the confusion: > SPy is not a "compiler for Python". There are features of the Python language which will never be supported by SPy by design. Don't expect to compile Django or FastAPI with SPy.

Yeah but then don't say that SPy is a (interpreter and) compiler in the first place? Just say it's a interpreter.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#35

Post by rurban »

graemep wrote: post #29

The problem with this is that the main value of Python is its ecosystem. SPy aims to be able to import Python libraries, but also not implement all Python features. If you are not 100% compatible how can you reliably import libraries? SPy seems most likely to be more likely to be appealing as a more Pythonic alternative to Cython rather than a Python replacement.

I did a similar project, a typed perl. cperl. I could import most the modules, and did add types to some of the important modules. Eg testing was 2x faster. I needed typing patches for about 10% for most CPAN packages.

A type is a contract, not a hint!

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#36

Post by yorwba »

AmazingTurtle wrote: post #34
ForceBru wrote: post #26

Earlier quoted context omitted.

Reading the next sentence clears the confusion: > SPy is not a "compiler for Python". There are features of the Python language which will never be supported by SPy by design. Don't expect to compile Django or FastAPI with SPy.

Yeah but then don't say that SPy is a (interpreter and) compiler in the first place? Just say it's a interpreter.

It is a compiler. It is not a compiler for Python, because there are valid Python programs it can't compile and isn't intended to compile.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#37

Post by antocuni »

graemep wrote: post #29

The problem with this is that the main value of Python is its ecosystem. SPy aims to be able to import Python libraries, but also not implement all Python features. If you are not 100% compatible how can you reliably import libraries? SPy seems most likely to be more likely to be appealing as a more Pythonic alternative to Cython rather than a Python replacement.

hello, author of the blog post and author of SPy here.

> how can you reliably import libraries?

the blog post specifies it but probably not in great level of detail. Calling python libs from spy will go through libpython.so (so essentially we will embed CPython). So CPython will import the library, and there will be a SPyCPython interop layer to convert/proxy objects on the two worlds.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#38

Post by antocuni »

nickcw wrote: post #22

This looks like a very interesting approach bringing comptime to a static version of python. This version of comptime can then be used to define new types in the same way Zig does it. I absolutely hate the terminology though red/blue redshifting etc. Why do blue functions disappear when redshifting? If you red shift blue then it goes down in frequency so you might get green or red. Perhaps my physics brain is just ov…

hello, spy author here.

> Why do blue functions disappear when redshifting? If you red shift blue then it goes down in frequency so you might get green or red. Perhaps my physics brain is just over thinking it!

yes I think you are overthinking :). It's not meant to be accurate physics of course.

The usage of colors to distinguish comptime vs runtim code comes from PyPy: in that context, we used "green" and "red", and initial versions of SPy used the same convention.

Then someone pointed out that green/red is not colorblind friendly and so I changed it to blue.

Having actual colors for the two phases is VERY useful for visualization: e.g. we already have a "spy --colorize" command which shows you which parts are blue and which are red.

As for "redshifting": the AST "before" has a mixture of blue and red colors, while the AST "after" has only red nodes, thus the final AST is "more red" than the first one, that's why I chose that name.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#39

Post by setopt »

rurban wrote: post #35
graemep wrote: post #29

The problem with this is that the main value of Python is its ecosystem. SPy aims to be able to import Python libraries, but also not implement all Python features. If you are not 100% compatible how can you reliably import libraries? SPy seems most likely to be more likely to be appealing as a more Pythonic alternative to Cython rather than a Python replacement.

I did a similar project, a typed perl. cperl. I could import most the modules, and did add types to some of the important modules. Eg testing was 2x faster. I needed typing patches for about 10% for most CPAN packages. A type is a contract, not a hint!

> A type is a contract, not a hint!

In Python it is a hint.

Re: SPy: An interpreter and compiler for a fast statically typed variant of Python

#40

Post by srean »

leobuskin wrote: post #13
srean wrote: post #5

If you want different parts of your code to be a statically typed Python lookalike Cython is a mature option

Yes, it's mature, but you (and your potential audience) basically need to learn a new language, a lot of quirks and "weird" (I'd even say counter-intuitive) nuances, and it's also significantly less readable in comparison with strict and typed Python. Even its modern syntax doesn't click immediately (also performance wise the new syntax somehow is a bit slower in my tests)

I am by no means a Cython fanboy but I think you are exaggerating the syntactic differences and readability differences.

Apart from type annotation they are very minor, well worth the speed benefits and type-error benefits. Given that we are discussing it in the context of SPy, SPy is not fully compatible with Python either, which is quite understandable and in my opinion a Good trade-off.

The benchmarking features are great, interactivity with C libraries is great.

One annoyance I have with Cython though is debuggability. But it's an annoyance, not a show-stopper.

Have used in production without problems.

Post reply on HN