Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node.js and Python Interoperability

fridgerator.github.io

Post reply on HN
Print view View original

11–20 of 22 posts

Re: Node.js and Python Interoperability

#11

Post by samcday »

Birch-san wrote: post #8
santa_boy wrote: post #7

I'm having the requirement to work with Node and Python now. I write my webapps in node but have data analytics scripts written in python to be invoked. I'm planning to use [python-shell - npm]( https://www.npmjs.com/package/python-shell )

Have you considered GraalVM? You can run both languages inside the same managed runtime (JVM), sharing the same data structures, inlining across language boundaries, and debug both with Chrome Dev Tools.

How much have you used GraalVM? I'm curious to know if it really works like that in practice. I've read what it says on the tin a few times and been impressed, but haven't actually sat down and played with it yet. The skeptic in me thinks that in a real-world production scenario, such a thing is surely a house of cards that falls down all the time in unexpected and difficult to diagnose ways. It'd be great to hear if there's people who can vouch and say it actually delivers on its promises of a polyglot utopia :)

Re: Node.js and Python Interoperability

#12

Post by agibsonccc »

Birch-san wrote: post #8
santa_boy wrote: post #7

I'm having the requirement to work with Node and Python now. I write my webapps in node but have data analytics scripts written in python to be invoked. I'm planning to use [python-shell - npm]( https://www.npmjs.com/package/python-shell )

Have you considered GraalVM? You can run both languages inside the same managed runtime (JVM), sharing the same data structures, inlining across language boundaries, and debug both with Chrome Dev Tools.

"Analytics" generally means "cython only". The numpy dep ends up being a problem.

Generally if you want python/java interaction, we maintain a tool called javacpp that handles this, we even bundle cpython: https://github.com/bytedeco/javacpp-presets

GraalVM itself also depends on javacpp for a portion of its features (LLVM wrapper): https://github.com/oracle/graal/blob/315f5dcf69c2e73fd13a5f8...

I'd be happy to answer questions about the overlap of the 2. I can say I happily execute python scripts from our embedded python and even point the python execution at an anaconda distribution.

Re: Node.js and Python Interoperability

#13

Post by agibsonccc »

samcday wrote: post #11
Birch-san wrote: post #8

Earlier quoted context omitted.

Have you considered GraalVM? You can run both languages inside the same managed runtime (JVM), sharing the same data structures, inlining across language boundaries, and debug both with Chrome Dev Tools.

How much have you used GraalVM? I'm curious to know if it really works like that in practice. I've read what it says on the tin a few times and been impressed, but haven't actually sat down and played with it yet. The skeptic in me thinks that in a real-world production scenario, such a thing is surely a house of cards that falls down all the time in unexpected and difficult to diagnose ways. It'd be great to hear if…

Graal itself still has limitations with many libraries yet, the biggest one being having any library that heavily uses reflection. I'd look at some of the work red hat is doing with some of its java libraries and quarkus: https://developers.redhat.com/blog/2019/03/07/quarkus-next-g...

Re: Node.js and Python Interoperability

#14

Post by silviogutierrez »

swordfeng wrote: post #10

I was once trying to do something even crazier: exposing full python accessibility to node.js, using proxies. So that you can use python objects and functions as native javascript objects and functions. It is feasible because the languages and runtimes share a number of common designs. There were several tough things however. The first was error handling. Yes you can catch exceptions in native code and convert it to…

This is great. I wanted to do something similar so that I could render Django templates in native JSX.

Any idea which of those RPC libraries could help achieve this? I'd love a Django model instance that could be used seamlessly in a JSX template.

Re: Node.js and Python Interoperability

#15

Post by santa_boy »

samcday wrote: post #11
Birch-san wrote: post #8

Earlier quoted context omitted.

Have you considered GraalVM? You can run both languages inside the same managed runtime (JVM), sharing the same data structures, inlining across language boundaries, and debug both with Chrome Dev Tools.

How much have you used GraalVM? I'm curious to know if it really works like that in practice. I've read what it says on the tin a few times and been impressed, but haven't actually sat down and played with it yet. The skeptic in me thinks that in a real-world production scenario, such a thing is surely a house of cards that falls down all the time in unexpected and difficult to diagnose ways. It'd be great to hear if…

Thanks for the input.

I remember coming across GraalVM. However, as I solo developer, I have found learning another new thing to be detrimental to my ability to getting things done. I have wasted way too many hours learning new frameworks, languages, platforms, etc.

I'm doing my best to avoid learning something new except when a strong compatibility with my current stack exists and is quickly implementable.

Re: Node.js and Python Interoperability

#16

Post by coldtea »

rcfox wrote: post #4
haolez wrote: post #3

Earlier quoted context omitted.

Tcl is one of those languages that feel special once you get to know it. In a good way! It’s like a command prompt for actual code. I don’t know how to explain it any better.

I'm not sure I got to know Tcl properly . I needed to use it to interact with an electronics design tool in order to extract information about a design, and so most of what I learned was from the tool's documentation about how to use its API, and some random Stack Overflow for specific questions. Anyway, Tcl felt very much like an 80's interpreted language to me. It seemed like every data structure was built on top o…

>It seemed like every data structure was built on top of strings, and every statement was being run through eval().

That's the case, and that's the whole magic...

Re: Node.js and Python Interoperability

#18

Post by masklinn »

swordfeng wrote: post #10

I was once trying to do something even crazier: exposing full python accessibility to node.js, using proxies. So that you can use python objects and functions as native javascript objects and functions. It is feasible because the languages and runtimes share a number of common designs. There were several tough things however. The first was error handling. Yes you can catch exceptions in native code and convert it to…

> The third was that js has no operator overloading, so I had to use .__add__() for example to call the python add operator.

I expect this wouldn't have worked in the long run as these methods are often just part of the protocol e.g. even `a == b` will try `type(a).__eq__(a, b)` then fall back to `type(b).__eq__(a)` (~~and then it'll do some weird stuff with type names IIRC~~[0]).

And most operators are not considered symmetric so the fallback is not the same as the initial (even `+` has `__add__` and `__radd__`, also `__radd__` might be called first depending on the relationship between type(a) and type(b)).

And then there's the "operations" which fallback to entirely different protocols e.g. `in` will first try to use `__contains__`, if that doesn't exist it uses `iter()` which tries to use `__iter__` but if that doesn't exist it falls back to calling `__getitem__` with non-negative sequential integer.

Which is why sometimes you define `__getitem__` for a pseudo-mapping convenience and then you get weird blowups that it's been called with `0` (you only ever expected string-keys). Because someone somewhere used `in` on your object and you hadn't defined `__iter__` let alone `__contains__`.

Good times.

[0] I misremembered: it's for ordering (not equality) in Python 2[1] `a [1] where there's always an ordering relationship between two objects, one of the things I'm most graceful Python 3 removed even if it's sometimes inconvenient

Re: Node.js and Python Interoperability

#20

Post by toyg »

agibsonccc wrote: post #13
samcday wrote: post #11

Earlier quoted context omitted.

How much have you used GraalVM? I'm curious to know if it really works like that in practice. I've read what it says on the tin a few times and been impressed, but haven't actually sat down and played with it yet. The skeptic in me thinks that in a real-world production scenario, such a thing is surely a house of cards that falls down all the time in unexpected and difficult to diagnose ways. It'd be great to hear if…

Graal itself still has limitations with many libraries yet, the biggest one being having any library that heavily uses reflection. I'd look at some of the work red hat is doing with some of its java libraries and quarkus: https://developers.redhat.com/blog/2019/03/07/quarkus-next-g...

IIRC you also end up relying on a graal-provided version of python that may or may not be kept up-to-date by a project for which it's just a secondary target. Been there with IronPython and Jython, it's no fun.
Post reply on HN