Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraalPy – A high-performance embeddable Python 3 runtime for Java

graalvm.org

Post reply on HN
Print view View original

91–100 of 151 posts

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#91

Post by antod »

commodoreboxer wrote: post #70
wenc wrote: post #54

Earlier quoted context omitted.

As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning) Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 ye…

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

Hard agree on the global Ruby import issues. I remember inspecting large custom Rails or Capistrano codebases in pry and having thousands of names imported. That and monkey patching had me wishing for Python with imports only having module scope and being a lot more explicit.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#92

Post by nextos »

commodoreboxer wrote: post #70
wenc wrote: post #54

Earlier quoted context omitted.

As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning) Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 ye…

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

It's a shame Python has a strong anti-FP stance with crippled lambdas. And an OO system that looks like it has been bolted in, compared to Ruby which is essentially a Smalltalk with Perl-like syntax and some Lisp influence.

These two issues would have been quite easy to fix and would have led to a completely different development experience. Python had a good implementation with a nice C FFI (CPython) right from the beginning, whereas Ruby MRI had lots of efficiency issues with long-running computations. IMHO this is one of the reasons why Python won. Building a numerics stack on top of MRI did not look very promising.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#93

Post by pansa2 »

nextos wrote: post #92
commodoreboxer wrote: post #70

Earlier quoted context omitted.

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

It's a shame Python has a strong anti-FP stance with crippled lambdas. And an OO system that looks like it has been bolted in, compared to Ruby which is essentially a Smalltalk with Perl-like syntax and some Lisp influence. These two issues would have been quite easy to fix and would have led to a completely different development experience. Python had a good implementation with a nice C FFI (CPython) right from the…

> an OO system that looks like it has been bolted in, compared to Ruby

I think the two languages just have different design philosophies. In Python, functions are fundamental and classes are built on top of them. In Ruby, objects are fundamental and functions (i.e. Procs etc) are themselves objects.

You could just as well claim that in Ruby, functions look like they have been bolted in. For example, you can’t call a Proc itself but need to call one of its methods.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#94

Post by pansa2 »

Myrmornis wrote: post #84
commodoreboxer wrote: post #70

Earlier quoted context omitted.

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

I think your comment needs to mention that Python has syntax for type annotations and two mature type checkers (mypy and pyright) with more under development. Python is thus very much part of the modern statically typed languages scene (moreso than Go) whereas Ruby isn't at all. Many people wouldn't touch Python today if it weren't for this.

> two mature type checkers

I’ve never quite understood how this works. Surely a type system is absolutely fundamental to a language - how can you have multiple incompatible ones?

Do you need to choose a particular type checker for each project? Are you limited to only using third-party libraries that use the same type checker?

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#95

Post by upghost »

malux85 wrote: post #79
upghost wrote: post #78

Earlier quoted context omitted.

I'm assuming you mean "how well does JVM concurrency play with Python concurrency"? Python concurrency works perfectly well on its own, Java/Clojure concurrency works very well on its own, trying to pass multithreaded information across the JVM boundary to Python while bypassing the GIL will result in a segfault (Edit: but there are "with-gil" wrappers you can use to prevent that, at a slight performance hit). In pra…

This is pretty interesting, what's the benefit over using python so directly with java? I mean, is the overhead of having these as seperate services / processes too much? I'm not trying to provoke I'm genuinely curious about the use case. Also, what's the dev workflow like? When I'm coding python I basically live inside the debugger (a.k.a the carmark method), do you use an IDE that understands both java and python?…

That's a really interesting line of questioning! We have a mode called "embedded mode" where you run the python application first, THEN initialize the JVM and Clojure via the Python "javabridge" package. From there, you can start your Clojure REPL and experience both Clojure's IDE integrated REPL experience or the Python debugger, depending on how you set it up. This also allows you to run maximally complex Python applications and is the recommended approach for training ML robustly.

I also tend to live inside the debugger for some things but for other things I really enjoy the Clojure/lisp style "in editor evaluation" (where the result appears right after your cursor when you evaluate the S-expression).

The usescasses question is a good one. Python has some pretty good libraries. For one project, we have a (Clojure) ring server and GCP cloud resources. Using the Python GCP secret manager to access protected cloud resources allows me to have the same code in dev and prod with minimal configuration.

Also sometimes it's just political. Maybe your workplace is a Clojure/Java only shop -- in that case, sometimes you can make the case Python is "just a library" and get some cool toys, in other circumstances where its Python only you can at least dev using your lisp REPL.

So if that kind of thing sounds fun to you (and you like emacs) you'll like this. If that sounds like hell to you, then it is!! I really tried hard to optimize around "fun" for the API, but it's also really performant, and great fun for hacking.

In particular I really love doing silly stuff with Python LLMs in the Clojure REPL.

So, tl;dr, I'd say it is really great if you are a certain kind of hacker who wants all the most fun toys and as an added bonus it also works in production.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#96

Post by pansa2 »

Myrmornis wrote: post #84
commodoreboxer wrote: post #70

Earlier quoted context omitted.

I agree with you, and I'll put it slightly stronger. Ruby is a better language than Python in every way except the very most important two: - Imports in Ruby seriously suck compared to Python. Everything requires into a global scope and an ecosystem like bundler which encourages centralizing all imports for your entire codebase into one file. - Python has docstrings encouraging in code documentation. Add common ecosy…

I think your comment needs to mention that Python has syntax for type annotations and two mature type checkers (mypy and pyright) with more under development. Python is thus very much part of the modern statically typed languages scene (moreso than Go) whereas Ruby isn't at all. Many people wouldn't touch Python today if it weren't for this.

> Python is thus very much part of the modern statically typed languages scene (moreso than Go)

Python’s type system is substantially more complex than Go’s - it’s probably more complete, but given it’s optional nature, less sound.

In “modern” type systems, is completeness considered more important than soundness? The success of TypeScript suggests it is.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#97

Post by upghost »

malux85 wrote: post #79
upghost wrote: post #78

Earlier quoted context omitted.

I'm assuming you mean "how well does JVM concurrency play with Python concurrency"? Python concurrency works perfectly well on its own, Java/Clojure concurrency works very well on its own, trying to pass multithreaded information across the JVM boundary to Python while bypassing the GIL will result in a segfault (Edit: but there are "with-gil" wrappers you can use to prevent that, at a slight performance hit). In pra…

This is pretty interesting, what's the benefit over using python so directly with java? I mean, is the overhead of having these as seperate services / processes too much? I'm not trying to provoke I'm genuinely curious about the use case. Also, what's the dev workflow like? When I'm coding python I basically live inside the debugger (a.k.a the carmark method), do you use an IDE that understands both java and python?…

Oh sorry there are actually huge performance benefits over a services based approach, because you're using the same memory space instead of serializing. This is particularly enormous for ML and data science space because of the work Chris did on hyper efficient mapping zero copy mapping of numpy arrays to tech.ml.dataset tensors.

Not even GraalVM has that! Not yet, anyway.

So there's a lot of easy perfomance synergies over microservices, but I'm the kind of dev where I tend to prioritize fun over performance as long as it's "performant enough". Fortunately, Chris (author of libpython-clj) is an ex-Nvidia performance obsessed dev though so the performance there is on point.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#98

Post by maple3142 »

pansa2 wrote: post #93
nextos wrote: post #92

Earlier quoted context omitted.

It's a shame Python has a strong anti-FP stance with crippled lambdas. And an OO system that looks like it has been bolted in, compared to Ruby which is essentially a Smalltalk with Perl-like syntax and some Lisp influence. These two issues would have been quite easy to fix and would have led to a completely different development experience. Python had a good implementation with a nice C FFI (CPython) right from the…

> an OO system that looks like it has been bolted in, compared to Ruby I think the two languages just have different design philosophies. In Python, functions are fundamental and classes are built on top of them. In Ruby, objects are fundamental and functions (i.e. Procs etc) are themselves objects. You could just as well claim that in Ruby, functions look like they have been bolted in. For example, you can’t call a…

Isn't Python's functions are just objects with a __call__ method, and such objects has a syntax sugar allowed them to be called like a function.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#99

Post by kfrzcode »

wenc wrote: post #54
yosefk wrote: post #34

Earlier quoted context omitted.

The reasons for all this stuff having been developed in Python also make Python interesting right now, all by themselves. It did not happen by accident; this stuff was developed fairly recently and there was no shortage of mature languages to choose from.

As a former Perl hacker who started using Python in 2005, I saw Python ride several waves. (Numerical computation, data science, deep learning) Perl was the leading tool for scripting and text parsing. Python didn’t really supplant it for a long time — until people started writing more complicated scripts that had to be maintained. Perl reads like line noise after 6 months whereas I can look at Python code from 20 ye…

Pragmatic use of $LANGUAGE is a telltale sign of the wizened programmer; one who understands the use-case and solution set well enough to know when the tool fits.

I wrote Ruby when I got started because it was the most accessible and the Rails learning content was top notch. Now I use python when I need more than a few `bash` pipes to accomplish anything, but if I were to solve a capital-P Problem, of course the tool often chooses the project after constraints.

Re: GraalPy – A high-performance embeddable Python 3 runtime for Java

#100

Post by fastball »

pjmlp wrote: post #36
tannhaeuser wrote: post #10

I guess what makes Python interesting right now is the integration with ML toolchains, CUDA, Metal/MLX, pytorch, tensorflow, LLM encoders/decoders, etc. more than Python the language. But can GraalVM run those codes meaningfully when Python is merely used for glue code with the important bits implemented in native code?

I am willing to live with Python as the Lisp we deserve to have, on this AI wave, when it finally gets a proper JIT story we can rely on, regardless of the workload. Currently it is a mix and match of an herculean engineering effort mostly ignored by the community (PyPy), DSLs for GPGPUs, bunch of C and C++ libraries that people keep referring to as "Python" when any language can have similar bindings, jython, IronPy…

The problem with PyPy is that it doesn't support the C-API, which is required for all those other high performance libraries.

So you gain the perf of a JIT, while losing out on most everything else high-performance in the Python ecosystem.

Post reply on HN