Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pyjion – A Python JIT Compiler

trypyjion.com

Post reply on HN
Print view View original

81–90 of 126 posts

Re: Pyjion – A Python JIT Compiler

#81

Post by haberman »

dec0dedab0de wrote: post #80
haberman wrote: post #72

Python is in a situation where it is enormously popular, due to a winning aesthetic in syntax/ergonomics that appeals to both newbies and experienced programmers. And due to this success, there is a lot of demand and interest in speeding the language up and getting rid of the GIL. But paradoxically, this veneer of simplicity hides an incredible amount of complexity. You have to read hundreds of lines of CPython to un…

My gut instinct is that many of Python's quirks are the reason for it's popularity. Though I might not be understanding your suggestion. Could you give a specific example of something you would clean up if you were able to? I have the video in a tab but won't be able to watch until later.

I think "a + b" is a good example.

I really recommend the talk I linked (https://youtu.be/qCGofLIzX6g), it gets deep into this stuff. I don't think anyone benefits from "a + b" having so many special cases.

Re: Pyjion – A Python JIT Compiler

#83

Post by rowanG077 »

jhgb wrote: post #73
rowanG077 wrote: post #71

Earlier quoted context omitted.

No you don't get that. A tracing JIT is not capable of producing an executable like that under normal circumstances. If a certain path is not taken during execution it might not be compiled at all. The point of the first futurama projection is that you get a fully runnable executable that is semantically equivalent to running the original program in the interpreter. A JIT only produces what it sees during execution.…

Clearly PyPy has to produce executable code if it wants to jump into it. The CPU wouldn't understand if it were asked to jump into something that isn't executable code. > The point of the first futurama projection ...was to delight viewers with what would turn out to be a wonderful pilot episode?

It produces executable code ad hoc based on the dynamic environment. Not an executable. That's not the same thing.

Re: Pyjion – A Python JIT Compiler

#84

Post by ggrrhh_ta »

marmaduke wrote: post #77
haberman wrote: post #72

Python is in a situation where it is enormously popular, due to a winning aesthetic in syntax/ergonomics that appeals to both newbies and experienced programmers. And due to this success, there is a lot of demand and interest in speeding the language up and getting rid of the GIL. But paradoxically, this veneer of simplicity hides an incredible amount of complexity. You have to read hundreds of lines of CPython to un…

Have you seen the Python docs which have pages on the execution model and data model, with notes in implementation details? https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/datamodel.html

I guess you are trying to imply that those documents cover a formal-enough description of what the execution and data model is. Well, even for the section "names" it just forgets to say if imported and non-imported names all share the same encoding and which should it be - just a nitpick, but with less than 5 seconds. Do you know that there are some behavioral limitations of dict that arise from a specific optimization in the implementation in C of dictionary iterators? If you create a new python following those documents, it is possible that you will allow a perfectly reasonable behavior that would fail in most other python interpreters.

Re: Pyjion – A Python JIT Compiler

#85

Post by jreese »

peterhil wrote: post #12

Does it have GIL? IronPython and Jython does not. https://wiki.python.org/moin/GlobalInterpreterLock How about Numpy performance?

It's not an alternate runtime, so the GIL is still there and Numpy performance should be unchanged, though any pure-Python modules from Numpy could potentially benefit from the JIT speedup.

Re: Pyjion – A Python JIT Compiler

#86

Post by zatarc »

progmetaldev wrote: post #65
zatarc wrote: post #33

Earlier quoted context omitted.

I've tried it on Fedora: https://docs.microsoft.com/en-us/dotnet/core/install/linux-f... > The latest version of .NET that's available in the default package repositories for Fedora is .NET 5. Installing .NET 6 through the default package repositories is coming soon. For now, you'll need to install .NET 6 in one of the following ways: > Install the .NET SDK or the .NET Runtime with Snap. > Install the .NET SDK or the…

.NET 6 was just released this morning, so I'd assume it will take at least a little bit of time to get into the default package repositories.

Thats the point.

Re: Pyjion – A Python JIT Compiler

#87

Post by aw1621107 »

chrisseaton wrote: post #78
marmaduke wrote: post #76

Earlier quoted context omitted.

Python docs have pages on the execution model and data model, with notes in implementation details. What more do you want? https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/datamodel.html Reading these documents is not hard, and it’s been an enormous help over the years for writing effective efficient Python.

> What more do you want? Something formal, so I can actually reason about it and test it. These documents are just informal prose. Are they sound? I don't know. Do you? Does anyone? Does my implementation match what they say? Who knows. Does CPython even match it? Does anyone know?

Out of curiosity, how many languages would meet those criteria? Only one I can think of off the top of my head is CompCert's C dialect. Are there others?

Re: Pyjion – A Python JIT Compiler

#88

Post by chrisseaton »

aw1621107 wrote: post #87
chrisseaton wrote: post #78

Earlier quoted context omitted.

> What more do you want? Something formal, so I can actually reason about it and test it. These documents are just informal prose. Are they sound? I don't know. Do you? Does anyone? Does my implementation match what they say? Who knows. Does CPython even match it? Does anyone know?

Out of curiosity, how many languages would meet those criteria? Only one I can think of off the top of my head is CompCert's C dialect. Are there others?

It's a spectrum - some languages do it a lot better than others. Not having anything more than a conversational English description of what it does is definitely the lower end of the spectrum. I'm sure very few are doing it perfectly, but for example Java has a formal semantics.

Re: Pyjion – A Python JIT Compiler

#89

Post by nevermore »

chrisseaton wrote: post #78
marmaduke wrote: post #76

Earlier quoted context omitted.

Python docs have pages on the execution model and data model, with notes in implementation details. What more do you want? https://docs.python.org/3/reference/executionmodel.html https://docs.python.org/3/reference/datamodel.html Reading these documents is not hard, and it’s been an enormous help over the years for writing effective efficient Python.

> What more do you want? Something formal, so I can actually reason about it and test it. These documents are just informal prose. Are they sound? I don't know. Do you? Does anyone? Does my implementation match what they say? Who knows. Does CPython even match it? Does anyone know?

As an outsider this may be your perception, but this is not how python's documentation works.

> These documents are just informal prose.

Not true. They're the language spec. Every guaranteed behavior of python is described clearly and concretely in these documents.

> Are they sound?

Yes.

> Does my implementation match what they say?

Yes.

> Does CPython even match it?

Yes.

> Does anyone know?

Yes! There's a very rigorous and thorough set of unit tests that specifically test an implementation's ability to match precisely the behavior described in these documents. All implementations (that I'm aware of, eg. cpython, pypy, jython, etc) state which versions of the spec they are compatible with, in other words they pass the unit test suite for that version.

Further, the maintainers of python (and by that I mean, regular contributors to the python-dev mailing list, not a cabal of robed individuals in a cave somewhere) are deeply aware of the language of the spec, the way the test suite implements it, and the importance of maintaining this relationship.

Re: Pyjion – A Python JIT Compiler

#90

Post by chrisseaton »

nevermore wrote: post #89
chrisseaton wrote: post #78

Earlier quoted context omitted.

> What more do you want? Something formal, so I can actually reason about it and test it. These documents are just informal prose. Are they sound? I don't know. Do you? Does anyone? Does my implementation match what they say? Who knows. Does CPython even match it? Does anyone know?

As an outsider this may be your perception, but this is not how python's documentation works. > These documents are just informal prose. Not true. They're the language spec. Every guaranteed behavior of python is described clearly and concretely in these documents. > Are they sound? Yes. > Does my implementation match what they say? Yes. > Does CPython even match it? Yes. > Does anyone know? Yes! There's a very rigor…

>> Are they sound?

> Yes.

That's great! Can you point me at the formal proof? I haven't seen it myself.

> There's a very rigorous and thorough set of unit tests that specifically test an implementation's ability to match precisely the behavior described in these documents.

How can you test against English prose? You can't. So someone's manually translated the prose into tests elsewhere I guess. Have they done that correctly? How can we verify that? Was there any ambiguity when they were interpreting the English?

It's easy to see where these simple English descriptions aren't covering everything. To give you a practical example - look at https://docs.python.org/3/reference/datamodel.html#object.__... - 'should return False or True' - what if it doesn't? Where's that specified? Is it somewhere else in this document? That's the kind of practical issue we work with when implementing languages.

Post reply on HN