Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraalPy – A high-performance embeddable Python 3 runtime for Java

graalvm.org

Post reply on HN
Print view View original

71–80 of 151 posts

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#71

Post by abirch »

smj-edison wrote: post #50
abirch wrote: post #24

Earlier quoted context omitted.

Minecraft Mods can only be written in Java and I want my kid to learn python. Jython is still 2.x and it'd be nice to let my kid write a minecraft mod in python. Not a business use case but a use case.

When I was learning programming, my coding class used a Bukkit plugin that connected to Python. I can't remember what it was called, but that was for Minecraft 1.7.10. Not sure if you were wanting Python specifically, but KubeJS lets you use JavaScript for mods. I think there's also a clojure integration.

Thank you. My 3rd grader knows basic python so I'd prefer to stick with that or Scratch

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#74

Post by rsyring »

wenc wrote: post #5

DuckDB is not currently a supported package, but Pandas and matplotlib are which is good. If DuckDB and Polars were supported and if they ran well, I suspect many data jobs could benefit.

Why would they benefit? When duckdb/Polaris are being used correctly, all the work is happening in the native stack. It should already be very fast compared to the Python runtime.

I recently moved a large ETL process that was mostly Python runtime processing to pyarrow/Polaris and wrote all the ETL logic in SQL. I've seen processes that used to take a week to run drop to about an hour (no exaggeration).

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#75

Post by wenc »

rsyring wrote: post #74
wenc wrote: post #5

DuckDB is not currently a supported package, but Pandas and matplotlib are which is good. If DuckDB and Polars were supported and if they ran well, I suspect many data jobs could benefit.

Why would they benefit? When duckdb/Polaris are being used correctly, all the work is happening in the native stack. It should already be very fast compared to the Python runtime. I recently moved a large ETL process that was mostly Python runtime processing to pyarrow/Polaris and wrote all the ETL logic in SQL. I've seen processes that used to take a week to run drop to about an hour (no exaggeration).

They wouldn’t benefit from performance because as you say they are already blazing fast as is. And I know what you mean — I rewrote a pure (granted old pre-2.0) pandas transformation into duckdb and compute time dropped from nearly an hour to single digit minutes.

But having these in Graal would allow more types of applications to be deployed in JVM stacks. As sibling comments note, many data science models are in python but production stacks are in Java.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#77

Post by wenc »

o11c wrote: post #68
wenc wrote: post #54

Earlier quoted context omitted.

As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning) Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 ye…

There was also the major anti-wave of Python 3. But it has managed to pull through despite ending up with broken strings (RIP all old code that needs to deal with legacy-encoded data), probably because there was no viable replacement.

Python 3 was a painful episode and I lingered on 2.7 and only ported over around 3.6.

But now 3.11 is fine again. Looking forward to faster releases.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#78

Post by upghost »

waldrews wrote: post #65
upghost wrote: post #62

FWIW we've had full Java/Python integration in Clojure for awhile now, courtesy of Chris Neurnberger and libpython-clj: https://github.com/clj-python/libpython-clj If you're into that sort of thing. Self-interest disclosure: I'm a major contributor and heavy user.

What's the GIL/threading story there?

I'm assuming you mean "how well does JVM concurrency play with Python concurrency"? Python concurrency works perfectly well on its own, Java/Clojure concurrency works very well on its own, trying to pass multithreaded information across the JVM boundary to Python while bypassing the GIL will result in a segfault (Edit: but there are "with-gil" wrappers you can use to prevent that, at a slight performance hit). In practice this tends not to be much of a problem as you setup a parallel workload on one side of the boundary or the other and pass information with a threadsafe queue. We do plenty of heavy parallel computations, data science, AI, fintech, etc.

There are certainly some leaky abstractions and there is a general expectation that you understand the quirks of Python and Clojure pretty well, so it's not for everyone. Knowing something about Java would probably help too but I've been using libpython-clj in production since 2017 years and I barely know anything about Java (compared to Python/Clojure).

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#79

Post by malux85 »

upghost wrote: post #78
waldrews wrote: post #65

Earlier quoted context omitted.

What's the GIL/threading story there?

I'm assuming you mean "how well does JVM concurrency play with Python concurrency"? Python concurrency works perfectly well on its own, Java/Clojure concurrency works very well on its own, trying to pass multithreaded information across the JVM boundary to Python while bypassing the GIL will result in a segfault (Edit: but there are "with-gil" wrappers you can use to prevent that, at a slight performance hit). In pra…

This is pretty interesting, what's the benefit over using python so directly with java? I mean, is the overhead of having these as seperate services / processes too much? I'm not trying to provoke I'm genuinely curious about the use case.

Also, what's the dev workflow like? When I'm coding python I basically live inside the debugger (a.k.a the carmark method), do you use an IDE that understands both java and python? Whats the debugging experience like? Can you set a breakpoint and then evaluate python code and expressions inside the debugger like you can if it was just solely a python project using VSCode and the python debugger?

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#80

Post by westurner »

kaashif wrote: post #16
westurner wrote: post #12

Earlier quoted context omitted.

"PEP 703 – Making the Global Interpreter Lock Optional in CPython" (2023) https://peps.python.org/pep-0703/ CPython built with --disable-gil does not have a GIL (as long as PYTHONGIL=0 and all loaded C extensions are built for --disable-gil mode) https://peps.python.org/pep-0703/#py-mod-gil-slot "Intent to approve PEP 703: making the GIL optional" (2023) https://news.ycombinator.com/item?id=36913328#36917709 https://…

This is pretty beside the point. The point is that X not having a GIL doesn't inherently mean Python on X also doesn't have a GIL.

CPython does not have a GIL Global Interpreter Lock GC Garbage Collection phase with --gil-disabled. GraalVM does have a GIL, like CPython without --gil-disabled.

How CPython accomplished nogil in their - the original and reference - fork is described in the topical linked PEP 703.

Post reply on HN