Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Unison language – a new approach to Distributed programming

unison-lang.org

Post reply on HN
Print view View original

81–90 of 116 posts

Re: The Unison language – a new approach to Distributed programming

#81

Post by actionfromafar »

zawodnaya wrote: post #56
pharmakom wrote: post #40

Earlier quoted context omitted.

Would this not work just as well with a lisp or even JS?

Well, it could be done in JS or Lisp. You'd have to replace all the references to dependencies in every function with a hash of the implementation of the referenced function and use some kind of global lookup table (which would need to be a distributed hash table or something). But this would be slow, so you'd need a compiler or some kind of processor to inline stuff and do CPS transforms, and by that time you've bas…

Or a beautiful, conceptually simple version of Unison. In any case, not more convoluted than for instance Javascript V8 already is to get the performance it has.

Re: The Unison language – a new approach to Distributed programming

#82

Post by stewoconnor »

dwohnitmok wrote: post #79
stewoconnor wrote: post #78

Earlier quoted context omitted.

It is up to the code handling the abilities to decide in which order to handle the abilities (or to handle them all at once). I can't think of any cases where it would make a difference in which order they were handled, however. Can you? I think perhaps it might in the case where abilities themselves were able to make requests of other abilities, but that's not something allowed by our type system currently

> I can't think of any cases where it would make a difference in which order they were handled, however. Can you? Presumably someAction : '{Choose, Abort} a if it's handled by `Choose.toList` and then `Abort.toOptional` in that order you end up with `Optional [a]` whereas if you do in the other order you have `[Optional a]` right? N.B. the reason this is theoretically important is that `someAction` may be written wit…

Thanks for the reply :)

> if it's handled by `Choose.toList` and then `Abort.toOptional` in that order you end up with `Optional [a]` whereas if you do in the other order you have `[Optional a]` right?

I mean, not necessarily. Something that handles a `'{Abort} a` doesn't necessarily produce an `Optional a`. It could produce a Boolean, it could produce an Int, whatever. This is really up to the handler. But I still don't see how you produce a "bug" because you can dispatch the requests to abilities in either order. You couldn't, for example, ever produce a (well-typed) situation where a call to abort doesn't abort, or a call to Choose.toList would fail to produce a list. (Perhaps if toList were allowed to also use {Abort} but it is not)

Re: The Unison language – a new approach to Distributed programming

#83

Post by waynesonfire »

brabel wrote: post #10

I think they are making a mistake that's common in this sort of project: trying too many new things at once! They already have a very innovative way of managing source code, with a database of definitions that keeps the hash of the syntax tree instead of actual source. That's a very neat idea that solves many problems (read their docs to understand why). But instead of developing that well enough so that it works wit…

> read their docs to understand why

hard pass. but thanks for the suggestion. I'll start with the home page and the examples and most likely end there.

Re: The Unison language – a new approach to Distributed programming

#84

Post by waynesonfire »

pharmakom wrote: post #40
zawodnaya wrote: post #35

Earlier quoted context omitted.

Well, there is a relationship. The relationship is specifically that Unison nodes can communicate code with one another unambiguously by exchanging hashes.

Would this not work just as well with a lisp or even JS?

or Erlang.

Re: The Unison language – a new approach to Distributed programming

#85

Post by stewoconnor »

cochne wrote: post #65

I really like the idea of Unison, but unfortunately when I went to try it out, it was much slower than even Python. I hope they can make it performant!

In one of our most recent blog posts we talked about the progress we are making on just-in-time native compilation:

https://www.unison-lang.org/blog/jit-announce/

We are expecting to be a monumental speedup for us. We have some promising results so far, but we haven't yet ported all of the runtime.

Re: The Unison language – a new approach to Distributed programming

#86

Post by tasuki »

IshKebab wrote: post #47
zawodnaya wrote: post #31

Earlier quoted context omitted.

What's an example of a pretty language?

I really hate the language but I have to say Python does at least look quite nice.

Yes it does look nice, but I wonder how you'd do eg pattern matching on ADTs...

Re: The Unison language – a new approach to Distributed programming

#87

Post by rlmark »

FpUser wrote: post #66

' - is used to denote a delayed computation. This is hard to see / easy to miss. What it this trend to abbreviate everything down to a shortest. What is wrong with delayed keyword for example

(Unison dev here) I hear you! :) `'` is easy to miss so we added the `do` keyword, and at least a few of us want to substitute `delay` as the keyword. Fortunately, we should be able to do that programmatically at the codebase level if the time comes for that, so folks won't have to hunt for their single quotes to change it everywhere.

Re: The Unison language – a new approach to Distributed programming

#88

Post by stewoconnor »

actionfromafar wrote: post #80
stewoconnor wrote: post #73

Earlier quoted context omitted.

(I'm the a Unison employee that works mostly on distributed computing) There are a few things about unison that make this easier: * content addressed code * unison can serialize any closure I can ask for a serialized version of any closure, and get back something that is portable to another runtime. So in a function, I can create a lambda that closes over some local variables in my function, ask the runtime for the c…

In a simple Lisp machine, such as something resembling PicoLisp, I can't see why not - iff instead of car and cdr being just linear addresses in a memory, have them itself be hashes. Since everything is made up from car and cdr, it's easy going from there. There is no difference between running locally, or anywhere. Just look up the data by request or gossip, as you said. (For performance reasons, one might want to l…

I'm not totally up to date on lisp and don't know anything about PicoLisp, so forgive me if there is stuff I'm missing :) but lemme try:

Lets say you wrote a imaginary program to sum a column in a csv:

(defun my-program () (let* ((raw-data) (s3-load-file "htpps://...")) ((parsed) (csv-parse raw-data)) ((column1) (csv-column parsed 1)) (mean column1))

In this pretend program we are using some 3rd party s3 library to fetch some data, some other 3rd party CSV library to extract the data. Now I want to run this on some other remote node. I know I can just sent that sexp to the remote node and have it eval it, but that is only going to work if the right versions of the S3 and CSV library are already in that runtime.

I want to be able to write a function like:

(defun remote-run (prog) (....))

That can take ANY program and ship it off to some remote node and have that remote node calculate the result and ship the answer back. I don't know of some way in lisp you could ask the runtime to give you a program which includes your calculation and the s3 functions and the csv functions.

In unison, I can just say `(seialize my-program)` and get back a byte array which represents exactly the functions we need to evaluate this closure. When the remote site tries to load those bytes into their runtime, it will either succeed or fail with "more information needed" with a list of additional hashes we need code for, and the two runtimes can recurse through this until the remote side has everything needed to load that value into the runtime so that it can be evaluated to a result.

Then, of course, this opens us up to the ability for us to be smart in places and say "before you try running this, check the cache to see if anyone ran this closure recently and already knows the answer"

Re: The Unison language – a new approach to Distributed programming

#89

Post by stewoconnor »

zawodnaya wrote: post #43
muricula wrote: post #26

How does the runtime manage different levels of trust between clients? How does the language grapple with code injection vulnerabilities? If I want to be able to receive data from an untrusted entity, but not code, how do I make sure they're not submitting code to my server to unpack and execute?

You can simply expose an HTTP endpoint that receives the data. You wouldn't want to expose the internode protocol endpoint to the internet. That said, it wouldn't be as bad as it sounds. Unison is a purely functional language, so if you don't explicitly provide the ability to e.g. do arbitrary I/O, then other nodes will not be able to send you code that does I/O. It will not type-check.

Yes, we additionally have functions builtin to the runtime that let you evaluate a term before you evaluate it to make sure it doesn't call any "forbidden" functions.

So in our cloud runtime, we blacklist EVERY IO function, then we can in our cloud runtime give you back the ability to do, for example, http requests, but not any other network IO. We won't let you open arbitrary files, but we'll provide ephemeral / persistent block storage through some other runtime ability.

This could also be used to do something like blacklist functions with known security vulnerabilities to catch people that aren't applying their patches!

Re: The Unison language – a new approach to Distributed programming

#90

Post by fwip »

brabel wrote: post #10

I think they are making a mistake that's common in this sort of project: trying too many new things at once! They already have a very innovative way of managing source code, with a database of definitions that keeps the hash of the syntax tree instead of actual source. That's a very neat idea that solves many problems (read their docs to understand why). But instead of developing that well enough so that it works wit…

The reason for the source code structure is because of what it enables for distributed computing.

It makes it simple to transparently ship the code (not just data) around to any worker node. The point of Unison Cloud is to disappear the difference between AWS EC2 and Lambda.

Post reply on HN