Rewriting Rust

I explain how Rust missed the mark in [1].

[1]: https://gavinhoward.com/2024/05/what-rust-got-wrong-on-forma...

> - Comparing apples and oranges. For example, Box makes pinning trivial (you can just move in and out of Pin no problem), but oftentimes people new to Rust try to prematurely optimise and eliminate a single pointer lookup. If that's the case, were you really writing the same thing in JS and in Rust?

Pointer lookups are cheap-ish, but allocating can be extremely expensive if you do it everywhere. I've seen plenty of lazy, allocation & clone heavy rust code end up running much slower than the equivalent javascript. I assume for this reason.

But in this case, I couldn't get it working even when putting Box all over the place.

> What would happen in your JS implementation if two streams were awaited concurrently, one received a message, and the other had to be cancelled? What if one threw an exception? In Rust, you're forced to think about those things from the start. In JS, you're coding the happy path.

I implemented error handling in the javascript code. That was easy - since async generators in javascript support try-catch. Javascript doesn't support concurrent execution - so that problem doesn't exist there.

Did multithreading contribute to javascript being easier to write than rust? Who cares? I had a problem to solve, and javascript made that trivial. Rust made it a total nightmare.

I didn't know about the stream wrappers when I started coding this up. That was how I eventually found an the answer to this problem: I read that code then adapted their approach.

And by the way, have you read the code in those wrappers? Its wild how they glue manual Future implementations and async functions together (with some clever Boxes) to make it work. It blew my mind how complex this code needs to be in order for it to work at all.

> Over two years of writing embedded, web, and CLI rust I didn't have to write a raw future once.

I'm happy for you, and I wish I had the same experience. Streams are bread and butter for my work (CRDTs, distributed systems and collaborative editing). And at this rate? Proper support for streams in rust is probably a decade away.

These are actually great and I would gladly welcome these changes if they were implemented in stable.

My wishlist:

* allow const fns in traits

* allow the usage of traits in const exprs. This would allow things like using iterators and From impls in const exprs, which right now is a huge limitation.

* allow defining associated type defaults in traits. This can already be worked around using macros somewhat effectively (see my supertrait crate) but real support would be preferable.

* allow eager expanding of proc macro and attribute macro input, perhaps by opting in with something like `#[proc_macro::expand(tokens)]` on the macro definition. Several core "macros" already take advantage of eager expansion, we peasants simply aren't allowed to write that sort of thing. As a side note, eager expansion is already possible for proc and attribute macros designed to work _within proc macro crates_, for example this which I believe is the first time this behavior was seen in the wild: https://github.com/paritytech/substrate/blob/0cbea5805e0f4ed...

* give build.rs full access to the arguments that were passed to cargo for the current build. Right now we can't even tell if it is a `cargo build` or a `cargo doc` or a `cargo test` and this ruins all sorts of opportunities to do useful things with build scripts

* we really need a `[doc-dependencies]` section in `Cargo.toml`

* give proc macros reliable access to the span / module / path of the macro invocation. Right now there are all sorts of projects that hack around this anyway by attempting to locate the invocation in the file system which is a terrible pattern.

* allow creating custom inner attributes. Right now core has plenty of inner attributes like `#![cfg(..)]` etc, and we have the syntax to define these, we simply aren't allowed to use custom attribute macros in that position

* revamp how proc macro crates are defined: remove the `lib.proc-macro = true` restriction, allowing any crate to export proc macros. Facilitate this by adding a `[proc-macro-dependencies]` section to `Cargo.toml` that separately handles proc-macro-specific dependencies. Proc macros themselves would have access to regular `[dependencies]` as well as `[proc-macro-dependencies]`, allowing proc macro crates to optionally export their parsing logic in case other proc macro crates wish to use this logic. This would also unblock allowing the use of the `$crate` keyword within proc macro expansions, solving the age old problem of "how do I make my proc macro reliably refer to a path from my crate when it is used downstream?"

* change macro_rules such that `#[macro_export]` exports the macro as an item at the current path so we can escape from this ridiculousness. Still allow the old "it exports from the root of the current crate" behavior, just deprecate it.

> Sandboxing against malicious crates is an out-of-scope problem. You can't do this at the language level; you need some combination of a verifier and runtime sandbox. WebAssembly components are a much more likely solution here. But there's lots of interest in having capabilities for other reasons, for things like "what allocator should I use" or "what async runtime should I use" or "can I assume the platform is 64-bit" or similar. And we do want sandboxing of things like proc macros, not because of malice but to allow accurate caching that knows everything the proc macro depends on - with a sandbox, you know (for instance) exactly what files the proc macro read, so you can avoid re-running it if those files haven't changed.

We've had a lot of talk about sandboxing of proc-macros and build scripts. Of course, more declarative macros, delegating `-sys` crate logic to a shared library, and `cfg(version)` / `cfg(accessible)` will remove a lot of the need for user versions of these. However, that all ignores runtime. The more I think about it, the more cackle's "ACLs" [0] seem like the way to go as a way for extensible tracking of operations and auditing their use in your dependency tree, whether through a proc-macro, a build script, or runtime code.

I heard that `cargo-redpen` is developing into a tool to audit calls though I'm imagining something higher level like cackle.

[0]: https://github.com/cackle-rs/cackle

Every single JS future is boxed. Moreover, they aren't just boxed, they are often backed by a hashmap (which may or may not be optimised away by the JIT). Elaborate allocation-free async is not an apple-to-apples comparison.

JS does support concurrent execution, Promise.all is an example. Without it, JS async would make little sense. The problem very much exists there, and try-catch is only a surface-level answer. As you can see here [1], the interaction of cancellation and async in JS is at least just as (or more) complex than in Rust.

By the way, multithreading has little to do with Pin. I presume you're thinking of Send bounds.

"To work at all" is very dismissive. Those wrappers are complex, but very well abstracted, well defined, and robust, the complexity is essential. Again, look at [1], JS async is hardly less complex, but also much more vague and ill-defined.

[1]: https://github.com/whatwg/streams/issues/1255