Live data from Hacker News

Modifying the Python object model

lwn.net

41–45 of 45 posts

Re: Modifying the Python object model

#41
post #36

Earlier quoted context omitted.

Pypy doesn't work well with cpython modules. That makes it a non-starter, sorry.

We have been addressing that for the last 3 years. Which CPython modules it does not work with for you? That comment was about something slightly different though - even though richards is 40x faster, pypy still is not faster on some workloads (like instagram in their measurments), which makes me think that there are other forces at play.

Well, right now, I simultaneously want to use the python-gobject stuff, Pandas, numpy, bcolz, and other assorted parts of the scipy universe.

Re: Modifying the Python object model

#42
post #38

Earlier quoted context omitted.

I guess the TurboFan backend of V8 is so powerful now that my guess is that it would be prepared to receive a Python or Lua frontend to it. Current Javascript is very complex, so maybe the current V8 Turbofan backend would already be fit to process a Python bytecode carefully crafted to fit this particular JIT backend?

From what I've seen, V8 is extremely wed to the javascript language model. I don't think it even has a concept of numbers that aren't javascript "numbers". Unless of course you go towards wasm territory, which I don't think is a territory particularly fun for dynamic languages.

WASM targets the TurboFan backend the same one the JS bytecode now does. So the backend can perfectly deal with a typed language like TS or Dart.

I think even in the frontend, the compiler try to guess the type beforehand, and anotate the real type for the compiler to optimize for the right type.

The problem is that i didnt dive that deep to know if a language as Python would be a good fit, nor im a "compiler guy" myself, so..

But i would love to do this as a backside project. The problem is my time is currently all taken by a big project.

But i would love to try to do this plug.. Thats why im winking here on HN. Maybe others also find a interesting thing to try themselves.

Re: Modifying the Python object model

#43
post #37

Earlier quoted context omitted.

Which is why I'm very excited about Graal Python ( https://github.com/graalvm/graalpython/blob/master/README.md ) It has the potential to bring an underlying framework which is built on industrial quality VM+JIT and it's primary goal is being compatible with the Scipy ecosystem at least ...Which is reason enough for unlimited optimism.

> industrial quality VM+JIT I'll skip over this mostly, just wondering what exactly you mean about this and whether you consider LLVM not to be "industrial quality" seeing as the failed Unladen Swallow project based itself upon that and it didn't seem to get them anywhere. > it's primary goal is being compatible with the Scipy ecosystem at least Well... it's not like PyPy isn't "compatible" with the scipy ecosystem.…

Llvm can't be used. I wish it could - http://doc.pypy.org/en/release-2.4.x/faq.html#could-we-use-l...

I'm not sure about your point. There is a new Python version built on Graal, that promises to largely maintain C compatibility (like graal Ruby) and still deliver performance.

I wish Pypy were getting funded by someone and have a lot of respect for what those guys achieved...But the fact remains that it is not bring used. Maybe graal Python can change that.

I used the words "industrial quality" instead of "pioneering" or "innovative". I think it's accepted that the millions of man hours spent on the JVM has made it one of the most incredible VM anywhere - is the defacto foundation on top of which you build big data (spark/hadoop), language theory (scala, closure, kotlin) and a billion mobile phones.

Re: Modifying the Python object model

#44
post #40
post #5

Earlier quoted context omitted.

The development of V8 was paid for by Google, and at that point they wanted to achieve market dominance against other big players. It seems nobody is willing to put big enough money behind making Python much faster. My view is that the limitations are almost purely financial (as in, paying heavily somebody as skillful as e.g. Mike Pall or Lars Bak(1) and his team), not technical. If Guido would not accept the "faster…

> My view is that the limitations are almost purely financial Your view is wrong. Implementing highly optimized versions of different languages (which may from the layman's point of view look quite similar) is not completely comparable. Python is a much more dynamic language than javascript - huge amounts of the language are overridable object by object, even down to attribute access. Hell, even down to isinstance()…

> Particularly when they don't seem to have significantly investigated the work of a team that's been working on the "fast python" problem for the last 15 years.

If you mean PyPy, PyPy was from the start intentionally "meta": it was never made to be simply a Python with a faster interpreter(s) and fast JIT(s) working in sync, like the modern Javascript engines work. It was intentionally a reimplementation of Python interpreter and all possible code in Python (that's where the name PyPy comes from), and then what's "optimized" is everything together: the new interpreter and the new Python implementation written in Python, all while generating a real C source. Which is then compiled as a normal C.

It's obviously too "meta" goal compared to the approaches used by fast Javascript engines, even if they also use some Javascript implementations for some library functions. When your starting goal is to "do everything in Python" you have already blocked yourself from being able to take the really best possible approach on every level of the engine. And even being meta enough, they still write:

"in code like this with a string-valued foo() function:"

   for x in mylist:
       s += foo(x)
"the JIT cannot optimize out intermediate copies. This code is actually quadratic in the total size of the mylist strings due to repeated string copies of ever-larger prefix segments."

So yes, there's definitely a room to make a faster usable implementation compared to the current PyPy. But it is a hard work, and needs a very focused and knowable leader(s), willing to take the "harder" approaches on every level, when needed. "Harder" than "we'll do everything in Python and only then optimize the whole thing together."

Re: Modifying the Python object model

#45
post #38

Earlier quoted context omitted.

From what I've seen, V8 is extremely wed to the javascript language model. I don't think it even has a concept of numbers that aren't javascript "numbers". Unless of course you go towards wasm territory, which I don't think is a territory particularly fun for dynamic languages.

WASM targets the TurboFan backend the same one the JS bytecode now does. So the backend can perfectly deal with a typed language like TS or Dart. I think even in the frontend, the compiler try to guess the type beforehand, and anotate the real type for the compiler to optimize for the right type. The problem is that i didnt dive that deep to know if a language as Python would be a good fit, nor im a "compiler guy" my…

I never said that it is inappropriate for a "typed language", in fact, far from it.
Post reply on HN