Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraalPy – A high-performance embeddable Python 3 runtime for Java

graalvm.org

Post reply on HN
Print view View original

81–90 of 151 posts

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#81

Post by rsyring »

wenc wrote: post #75
rsyring wrote: post #74

Earlier quoted context omitted.

Why would they benefit? When duckdb/Polaris are being used correctly, all the work is happening in the native stack. It should already be very fast compared to the Python runtime. I recently moved a large ETL process that was mostly Python runtime processing to pyarrow/Polaris and wrote all the ETL logic in SQL. I've seen processes that used to take a week to run drop to about an hour (no exaggeration).

They wouldn’t benefit from performance because as you say they are already blazing fast as is. And I know what you mean — I rewrote a pure (granted old pre-2.0) pandas transformation into duckdb and compute time dropped from nearly an hour to single digit minutes. But having these in Graal would allow more types of applications to be deployed in JVM stacks. As sibling comments note, many data science models are in py…

> But having them this would allow more types of applications to be deployed in JVM stack...

Ah...makes sense now. I was thinking along the lines of someone switching to the JVM for better performance, but being held back by the absence of those libraries.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#82

Post by lordgroff »

BiteCode_dev wrote: post #44
yosefk wrote: post #34

Earlier quoted context omitted.

The reasons for all this stuff having been developed in Python also make Python interesting right now, all by themselves. It did not happen by accident; this stuff was developed fairly recently and there was no shortage of mature languages to choose from.

The people disliking the language are very vocal about it, but there is a huge amount of silent people that loves it and an even bigger amount that just like it as much as alternatives. It's mainstream now, not trending like 10 years ago, so there is no hype about it anymore. We just use it to do stuff. Add to that the existing excellent ecosystem, the strong culture of scientific stacks and a very good story for pro…

I mean, even on HN, I'd say if there's derision, it's mostly one uttered with a yawn rather than genuine hate. And that's almost justified; while I spend a lot of my time with lots of different languages (I can't think of a single one I outright hate btw), Python is the one that pays for my things and... Well, there's not much drama there is there (now that we're lost 2->3 anyway)? It's a glue language that's easy to learn, but offers tons of depth should you want it. My primary annoyance at Python used to be the typing, but type annotations have made this less of an issue. It's a nice language and you can do almost everything with it. It's a bit boring, but I guess that's a good thing.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#83

Post by kaashif »

westurner wrote: post #80
kaashif wrote: post #16

Earlier quoted context omitted.

This is pretty beside the point. The point is that X not having a GIL doesn't inherently mean Python on X also doesn't have a GIL.

CPython does not have a GIL Global Interpreter Lock GC Garbage Collection phase with --gil-disabled. GraalVM does have a GIL, like CPython without --gil-disabled. How CPython accomplished nogil in their - the original and reference - fork is described in the topical linked PEP 703.

Yes, I know. What I'm saying is that:

It's possible to have a language that doesn't have a GIL, which you implement Python in, but that Python implementation then has a GIL.

The point being that you can't say things like: Jython is written in Java so it doesn't have a GIL. CPython is written in C so doesn't have a GIL. And so on.

If this isn't clear, I apologize.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#84

Post by Myrmornis »

commodoreboxer wrote: post #70
wenc wrote: post #54

Earlier quoted context omitted.

As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning) Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 ye…

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

I think your comment needs to mention that Python has syntax for type annotations and two mature type checkers (mypy and pyright) with more under development. Python is thus very much part of the modern statically typed languages scene (moreso than Go) whereas Ruby isn't at all. Many people wouldn't touch Python today if it weren't for this.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#85

Post by RMPR »

theLiminator wrote: post #51
tln wrote: post #15

Earlier quoted context omitted.

Yes, apparently it can https://www.graalvm.org/dev/reference-manual/python/Native-E... > CPython provides a native extensions API for writing Python extensions in C/C++. GraalPy provides experimental support for this API, which allows many packages like NumPy and PyTorch to work well for many use cases. The support extends only to the API, not the binary interface (ABI), so extensions built for CPython are not binary…

I wonder if hpy will solve the extension problem.

While hpy is great and I'm excited about it, I would rather bet on the limited C API[0] (which is basically what hpy tries to be if I understand correctly).

0: https://devguide.python.org/developer-workflow/c-api/#limite...

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#86

Post by RMPR »

pjmlp wrote: post #36
tannhaeuser wrote: post #10

I guess what makes Python interesting right now is the integration with ML toolchains, CUDA, Metal/MLX, pytorch, tensorflow, LLM encoders/decoders, etc. more than Python the language. But can GraalVM run those codes meaningfully when Python is merely used for glue code with the important bits implemented in native code?

I am willing to live with Python as the Lisp we deserve to have, on this AI wave, when it finally gets a proper JIT story we can rely on, regardless of the workload. Currently it is a mix and match of an herculean engineering effort mostly ignored by the community (PyPy), DSLs for GPGPUs, bunch of C and C++ libraries that people keep referring to as "Python" when any language can have similar bindings, jython, IronPy…

> I am willing to live with Python as the Lisp we deserve to have

You can have your cake and eat it too https://github.com/hylang/hy

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#87

Post by westurner »

kaashif wrote: post #83
westurner wrote: post #80

Earlier quoted context omitted.

CPython does not have a GIL Global Interpreter Lock GC Garbage Collection phase with --gil-disabled. GraalVM does have a GIL, like CPython without --gil-disabled. How CPython accomplished nogil in their - the original and reference - fork is described in the topical linked PEP 703.

Yes, I know. What I'm saying is that: It's possible to have a language that doesn't have a GIL, which you implement Python in, but that Python implementation then has a GIL. The point being that you can't say things like: Jython is written in Java so it doesn't have a GIL. CPython is written in C so doesn't have a GIL. And so on. If this isn't clear, I apologize.

Oh okay. Yeah I would say that the Java GC and the ported CPython GIL are probably limits to the performance of any Python in Java implementation.

But are there even nogil builds of CPython C extensions on PyPi yet anyway.

Re: Ghidraal and various methods of Python in Java: https://news.ycombinator.com/item?id=36454485

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#88

Post by RMPR »

Myrmornis wrote: post #84
commodoreboxer wrote: post #70

Earlier quoted context omitted.

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

I think your comment needs to mention that Python has syntax for type annotations and two mature type checkers (mypy and pyright) with more under development. Python is thus very much part of the modern statically typed languages scene (moreso than Go) whereas Ruby isn't at all. Many people wouldn't touch Python today if it weren't for this.

> syntax for type annotations and two mature type checkers (mypy and pyright)

I would throw Pyre in there too

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#89

Post by RMPR »

toyg wrote: post #43
pvorb wrote: post #35

Earlier quoted context omitted.

Did you look into Jython back then?

Jython has historically lagged hard , often falling behind for very extended periods. For a time their releases basically just stopped, which led to them missing support for pretty much anything between 2.7 and 3.6 (iirc). I know the project basically rebooted at some point, but I've since lost interest.

Not to mention the biggest drawback imho. Those alternative implementations don't support C extensions.
Post reply on HN