Live data from Hacker News

Why asynchronous Rust doesn't work

eta.st

171–180 of 305 posts

Re: Why asynchronous Rust doesn't work

#171
"I’d like to make a simple function that does some work in the background, and lets us know when it’s done by running another function with the results of said background work"

This is why he's got a problem, in a nutshell. I suspect the author has a heavy Javascript background and is used to Continuation Passing Style (CPS)[1]. The problem is that in a non-interpreted language that is a horrible way to do things because of scope and references. A better approach is to use messages from your thread. In C++ you could do this with ZeroMQ inproc sockets [2], or other mechanisms. In JAVA you could use a number of methods, such as a ConcurrentLinkedQueue [3]. In Go, you have channels [4]. In Rust, you have..well, also channels. Rust By Example already has a similar example that uses channels [5]. The concept of a Message Passing Interface (MPI) works amazingly well, and there is at least one flavor implemented for all the major languages (and often more).

The only languages I've seen that get callback style interfaces done well are Clojure, Lisp, Erlang, and Elixir.

The only thing I've seen that's better than MPI, in some situations, is Tuple Spaces [6].

Tar and feather me for it, but I miss JINI [7] and JavaSpaces [8] when I have to do a system in JAVA. JINI is now Apache River [9], but the last release was 2016.

[1] https://en.wikipedia.org/wiki/Continuation-passing_style

[2] http://api.zeromq.org/2-1:zmq-inproc

[3] https://docs.oracle.com/javase/9/docs/api/java/util/concurre...

[4] https://gobyexample.com/channels

[5] https://doc.rust-lang.org/rust-by-example/std_misc/channels....

[6] https://software-carpentry.org/blog/2011/03/tuple-spaces-or-...

[7] https://en.wikipedia.org/wiki/Jini

[8]https://river.apache.org/release-doc/current/specs/html/js-s...

[9] https://river.apache.org/

Re: Why asynchronous Rust doesn't work

#172

Earlier quoted context omitted.

If you are going to wrap everything in Rc, just use kotlin or c-sharp or python?

While I understand that with Graal and .Net you can ostensibly make native, static binaries for Kotlin or C#, I'm very skeptical that it works well in practice. In particular, I'm guessing it will feel like swimming upstream, fighting an ecosystem and build tooling which mostly assume you're running on a VM. And then there's the question of performance... And of course, with Python your code will run 100x slower than…

With the modern JVM you actually have to work quite hard to write native code (Rust/C++/C) that outperforms an equivalent Java/Kotlin/Scala implementation. And it is quite easy to perform worse with a native implementation. Of course that is subject to various caveats:

1. Anything running on the JVM will need a 50-500ms of startup time.

2. The JVM implementation will not reach max performance until the runtime has optimized and JIT'd the relevant code paths.

3. There is memory overhead of the VM itself so your runtime will (all else equal) probably require more memory.

If you do need to use Graal to generate native images though it can actually be quite nice. You can run locally (or in benchmarking environments) on the VM and get all the tooling and metrics that come along with that, but build a native binary to actually deploy. I agree that it can be kind of a pain though.

Re: Why asynchronous Rust doesn't work

#173
post #166

Earlier quoted context omitted.

It's not that I don't see the benefit of the CPU doing something useful when waiting for I/O, my confusion comes from the fact that people like to express this using promises/await. Why not just arrange it like this? function non_awaited(cache, db, metrics) { let result = cache.query(...); if (!result) { result = db.query(...); cache.store(result); } metrics.log(...); return result; } Basically doesn't a good threadi…

I can't speak authoritatively, but I can think of some good reasons you might not want to automatically and implicitly await every invocation of an async function. As designed, calling an async function just returns a Promise, and any Promise can be awaited. This means that I can pass that Promise around, and it also means I can use a Promise-based library (of which there are many) easily from within my async code. A…

All right more flexibility, good point.

However what languages are you thinking of when talking about this?

I've done a lot of Java programming where we commonly use threads and thread pools, although there are non blocking libraries out there too.

Is it common these days to mix promises and threads (waiting on a promise in a thread), or is what I just said nonsense?

Re: Why asynchronous Rust doesn't work

#174
post #73

Earlier quoted context omitted.

The article is also conflating synchronous single-threaded, synchronous multi-threaded and asynchronous programming. Each have their own usage, and no, a multi-threaded program is not the same as an asynchronous one. For example, using threads and channels instead of async/await is not a design flaw if your workload is mostly about large, blocking computations on a read-only shared state with no I/O. In that situatio…

its really strange that there are two languages running around together. one which is very opinionated in how to manage memory in a stack discipline and another which just uses reference counts. they don't quite mix. so you need to be aware of which one you're (implicitly using), and you may need library functions for both colors. you have to admit this adds some additional mental overhead. but what got me when tryin…

IME, using reference counts there is also not a good idea. In practice I think it's better to treat async tasks just like goroutines and then use channels to communicate.

Re: Why asynchronous Rust doesn't work

#175
post #70

Earlier quoted context omitted.

Then you are essentially introducing a verbose, unoptimized garbage collector?

And you still run into all sorts of ergonomic issues when you inevitably need to dereference your Rc pointer. That said, I sympathize with the parent's desire for a proper Rust-lite with a GC: C-family syntax, great tooling, great documentation, great standard library and ecosystem, native+static compilation by default. Of course, someone will ignore those criteria and come in suggesting OCaml/Reason...

Sounds like D, actually.

As D is making gc optional, I'm guessing rust will evolve ADT crates that makes it easier to do massive parallell processing via message passing. But like with C, I'm not sure most of that should be "part of the language" - might not be something you need for your bootsector or ABS break system controller...

Re: Why asynchronous Rust doesn't work

#176
post #160
post #155

Earlier quoted context omitted.

C++ lambdas are much better than Rust ones, because you can explicitly decide what to copy inside of the closure. As for the author being happier writing in a higher level language, it kind of proves the point that Rust's main target is the domain where any sort of automatic memory management aren't a viable option. Pushing Rust outside of this domain is only trying to fit a square peg into a round hole.

C++ lambdas need that feature because they don't have lifetimes or a borrow checker. Rust closures don't need that feature because rust has a borrow checker that ensures you aren't referencing something you shouldn't be. You can explicitly decide what to copy inside a Rust closure as well, you just use a `move` closure and create references for anything you need referenced outside the closure instead.

Move closures assume everything is movable, in C++ you can control what actually takes place, and yes there is the possibility to get it wrong.

Re: Why asynchronous Rust doesn't work

#177

Earlier quoted context omitted.

And you still run into all sorts of ergonomic issues when you inevitably need to dereference your Rc pointer. That said, I sympathize with the parent's desire for a proper Rust-lite with a GC: C-family syntax, great tooling, great documentation, great standard library and ecosystem, native+static compilation by default. Of course, someone will ignore those criteria and come in suggesting OCaml/Reason...

> Of course, someone will ignore those criteria and come in suggesting OCaml/Reason... I hoped Go to be such a language, but it failed to fulfill my needs by throwing away all the PL knowledge that humanity has accumulated for decades. I still mourn for the missed opportunity by Google.

Have you looked at D? (or perhaps zig - but I think D would be closer to a "sane" go).

Re: Why asynchronous Rust doesn't work

#178

Earlier quoted context omitted.

If you are going to wrap everything in Rc, just use kotlin or c-sharp or python?

Rust has advantages over C#, even for high level programming: * Rust traits are more flexible than C# interfaces, especially when combined with generics (implementing traits for foreign types, associated types, each method can have its own constraints, conditional trait implementation, #derive) * Rust has much stronger thread safety guarantees (absence of data races, preventing access to a mutex's data without lockin…

> Python is not an option for me, since I like static typing.

Have you checked out mypy recently (past few years)? With all the strict flags, it's pretty damn hard to get anything but bulletproof static typed code to pass. The only major downside is it lacks higher kinded types at the moment, and there is a PR in the works to add that. But it has all of the other accoutrements you'd expect from a modern type system: generics, structural subtyping, co/contravariance.

Re: Why asynchronous Rust doesn't work

#179
post #163

Earlier quoted context omitted.

One of the very particular "undebuggable" issues (safe) Rust solves is data races. Experience tells us that humans can't successfully reason about non-trivial concurrent programs unless they exhibit Sequential Consistency. In Rust you're promised this is what you get. Maybe what you wrote is stupid and wrong, but it has Sequential Consistency. "Oh," you exclaim during debugging, "A might happened before B and then we…

Only for data races via threads on the same process, it does nothing to prevent data races via shared memory using IPC mechanisms across processes.

How so? Obviously you can build an unsafe IPC mechanism with concurrent access, label it "safe" when it isn't, and then say "Look at this horrible mess, I blame Rust" but it seems like it'd be faster to just implement std::ops::Index unsafely and then blame Rust because thing[len+1] blew up even though Rust has "memory safety".

Now I'm going to write an amusing aside. One way you could get into this trouble is if your hardware allows arbitrary foreign memory writes as "IPC". The BBC microcomputer allowed this over the network! You could send a bunch of bytes over Econet (a 1980s network from Acorn Computers available for the BBC, Electron and Archimedes computers), addressed to another BBC micro on the network, asking they be written to a RAM buffer, and the remote hardware would do so. We used this to run a Multi-user Dungeon at school in about 1989 or so. A "server" ran the actual MUD software, and individual users signed in from a computer on the network around the school to play, when they typed a command their command buffer was transmitted over Econet, and then the server wrote remotely to their display RAM to show the result on their screen.

Fortunately your computer is not a BBC Microcomputer, and foreign processes do not (on the whole) get to scribble on your program's memory. So this should not be a problem unless you specifically make this unsafe decision in your Rust program.

Re: Why asynchronous Rust doesn't work

#180
post #166

Earlier quoted context omitted.

I can't speak authoritatively, but I can think of some good reasons you might not want to automatically and implicitly await every invocation of an async function. As designed, calling an async function just returns a Promise, and any Promise can be awaited. This means that I can pass that Promise around, and it also means I can use a Promise-based library (of which there are many) easily from within my async code. A…

All right more flexibility, good point. However what languages are you thinking of when talking about this? I've done a lot of Java programming where we commonly use threads and thread pools, although there are non blocking libraries out there too. Is it common these days to mix promises and threads (waiting on a promise in a thread), or is what I just said nonsense?

There was recently an article/discussion on async in ruby:

https://news.ycombinator.com/item?id=29049881

And I think the example is a good one for how doing parallell io can be simple with async tasks/promises:

  require "async"
  require "open-uri"

  Async do |task|
    task.async do
      URI.open("https://httpbin.org/delay/1.6")
    end

    task.async do
      URI.open("https://httpbin.org/delay/1.6")
    end
  end
(completes in ~time of slowest request, not sum of requests).

Sure, one could use threads/processes/green threads etc.

Post reply on HN