Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Rust

josephg.com

Post reply on HN
Print view View original

151–160 of 410 posts

Re: Rewriting Rust

#151

Post by josephg »

JoshTriplett wrote: post #26

> Now, there are issue threads like this, in which 25 smart, well meaning people spent 2 years and over 200 comments trying to figure out how to improve Mutex. And as far as I can tell, in the end they more or less gave up. The author of the linked comment did extensive analysis on the synchronization primitives in various languages, then rewrote Rust's synchronization primitives like Mutex and RwLock on every major…

Author here. Thanks for the in depth response. I appreciate hearing an insider's perspective.

> I always find it amusing to see, simultaneously, people complaining that the language isn't moving fast enough and other people complaining that the language is moving too fast.

I think people complain that rust is a big language, and they don't want it to be bigger. But keeping the current half-baked async implementation doesn't make the language smaller or simpler. It just makes the language worse.

> The main blocker for supporting partial borrows in public APIs has been how to expose that to the type system in a forwards-compatible way that supports maintaining stable semantic versioning

I'd love it if this feature shipped, even if it only works (for now) within a single crate. I've never had this be a problem in my crate's public API. But it comes up constantly while programming.

> Sandboxing against malicious crates is an out-of-scope problem. You can't do this at the language level; you need some combination of a verifier and runtime sandbox.

Why not?

If I call a function that contains no unsafe 3rd party code in its call tree, and which doesn't issue any syscalls, that function can already only access & interact with passed parameters, local variables and locally in-scope globals. Am I missing something? Because that already looks like a sandbox, of sorts, to me.

Is there any reason we couldn't harden the walls of that sandbox and make it usable as a security boundary? Most crates in my dependency tree are small, and made entirely of safe code. And the functions in those libraries I call don't issue any syscalls already anyway. Seems to me like adding some compile-time checks to enforce that going forward would be easy. And it would dramatically reduce the supply chain security risk.

Mind explaining your disagreement a little more? It seems like a clear win to me.

Re: Rewriting Rust

#152

Post by tonyedgecombe »

imron wrote: post #14
feelingsonice wrote: post #7

>And I don't know if it will ever be there. Progress on the language has slowed so much. When I first started using it, every release seemed to add new, great features in stable rust. Now? Crickets. Is frustration with Rust on the rise? I just started using Rust few month ago and absolutely love it. I can't tell what's going on with the Rust foundation so I can only judge by reading sentiments. Nothing would kill my…

“There are only two kinds of languages: the ones people complain about and the ones nobody uses.” ― Bjarne Stroustrup

Yeah, and C++ is morphing from one to the other.

Re: Rewriting Rust

#153

Post by leoedin »

anon-3988 wrote: post #105
zifpanachr23 wrote: post #85

Earlier quoted context omitted.

Because for a lot of companies, especially ones in industries that Rust is supposedly hoping to displace C and C++ in, dependencies are a much larger concern than memory safety. They slow down velocity way more than running massive amounts of static and dynamic analysis tools to detect memory issues does in C. Every dependency is going to need explicit approval. And frankly, most crates would never receive that appro…

Does C++ codebases with similar features parity somehow requires less code?

There's probably a similar amount of code in the execution path, but the Rust ecosystem reliance on dependencies means that you're pulling in vast amounts of code that doesn't make it to your final application.

A C++ library author is much more likely to just implement a small feature themselves rather than look for another 3rd party library for it. Adding dependencies to your library is a more involved and manual process, so most authors would do it very selectively.

Saying that - a C++ library might depend on Boost and its 14 million LOC. Obviously it's not all being included in the final binary.

Re: Rewriting Rust

#154

Post by lifthrasiir »

cmrdporcupine wrote: post #141
lifthrasiir wrote: post #79

Earlier quoted context omitted.

I consciously remove and rewrite various dependencies at work, but I feel it's only a half of the whole story because either 1K or 4M lines of code seem to be equally inaccurate estimates for the appropriate number of LoC for this project. It seems that most dependencies of cargo-watch are pulled from three direct requirements: clap, cargo_metadata and watchexec. Clap would pull lots of CLI things that would be natur…

It's frustrating because the grand-daddy of build systems with automatic transitive dependency management -- Maven -- already had tools from day one to handle this kind of thing through excluded dependencies (a blunt instrument, but sometimes necessary). In my experience, [patch] doesn't cut it or compare. That, and the maven repository is moderated. Unlike crates.io. Crates.io is a real problem. No namespaces, basic…

> In my experience, [patch] doesn't cut it or compare.

AFAIK what Maven does is an exclusion of dependency edges, which is technically an unsafe thing to do. Cargo [patch] is a replacement of dependency vertices without affecting any edges. (Maven surely has a plugin to do that, but it's not built-in.) They are different things to start with.

Also I believe that the edge exclusion as done by Maven is (not just "technically", but) really unsafe and only supported due to the lack of better alternatives. Edges are conceptually dependent to the incoming vertex, so it should be that vertex's responsibility to override problematic edges. An arbitrary removal of edges (or vertices) is much harder to track and many other systems have related pains from that.

What I'm proposing here is therefore the extension of Cargo's vertex replacement: you should be able to share such replacements so that they can be systematically dealt. If my transitive dependencies contain some crate X with two different versions 1.0 and 2.0 (say), I should be able to write an adapter from 2.0 to 1.0 or vice versa, and ideally such adapter should be available from the crate author or from the community. I don't think Maven did try any such systematic solution.

> That, and the maven repository is moderated. Unlike crates.io.

Only the central repository is moderated by Maven. Maven is not much better than Cargo once you have more remote repositories.

> Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about.

Namespace is not a solution for name squatting: namespace is just yet another identifier that can be squatted. If you are worried about squatting, the only effective solution is sandboxing, everything else is just moving the goal post.

The very existence of remote repositories also means that you can't always moderate all the stuffs and get rid of abandoned stuffs. You have to trust repositories, just like that you have to trust crates with crates.io today.

Re: Rewriting Rust

#155

Post by josephg »

olivierduval wrote: post #34

> Most crates I use - like human-size or serde don't need any special capabilities to work. So we don't need to worry so much about their authors "turning evil" and adding malicious code to our software well... :-( Actually, it's obvious that some authors might "turn evil" dumbly, by abusing some kind of priviledged permissions. By chance, these kinds of supply-chain risks are "easily" identified because 1) the permi…

Author here. Even if the permission system needs to be explicitly enabled and most people don't enable it, that would still have the effect of adding an early warning system for the entire rust ecosystem.

> The real problem is "how can human-size be used to subvert the program ?" For example: what is happening if the returned size "forget" or "add" 100 bytes to files bigger than 1 KB ? As a remininder, STUXNET was about some speed a tiny bit faster than planned and shown...

I read this argument in a similar vein to the argument against rust's unsafe blocks. "Look, C code will always need some amount of unsafe. So why bother sandboxing it?"

But in practice, having explicit unsafe blocks has been a massive win for safety in the language. You can opt out of it at any time - but most people never need to!

A 90% solution doesn't solve the problem entirely. But it does solve 90% of the problem. And thats pretty bloody good if you ask me! Sure - my safe rust decompression library could still maliciously inject code in files that it decompresses. But having checks like this would still reduce the security surface area by a huge amount.

Less implicit trust in random crate authors is a good thing. I don't want thousands of crate authors to be allowed to execute totally arbitrary code on my machine! The current situation is ridiculous.

Re: Rewriting Rust

#156

Post by josephg »

lifthrasiir wrote: post #129
tommiegannert wrote: post #75

> Rust doesn't have syntax to mark a struct field as being in a borrowed state. > ast_nodes: Vec , Oh, that would be neat to replace the https://github.com/tommie/incrstruct I wrote for two-phase initialization. Unlike Ouroboros and self_cell, it uses traits so the self-references can be recreated after a move. Whether it's a good idea, I don't know, but the magic Ouroboros applies to my struct feels wrong. But I say…

`if some_var.is_some_and(|x| some_expr)` is the current way to do that. It is less flexible and doesn't actually bind `x` into the conditional body (hence the proposal) but works today.

Yeah, but I almost always want to pattern match out the value when I do that.

As I said in the post you can also write this:

    if let (Some(x), true) = (my_option, expr) {
But then it doesn't short-circuit. (expr is evaluated in all cases, even when the optional is None).

Both approaches are also weird. It'd be much better to just fix the language to make the obvious thing work.

Re: Rewriting Rust

#157

Post by tonyedgecombe »

feelingsonice wrote: post #7

>And I don't know if it will ever be there. Progress on the language has slowed so much. When I first started using it, every release seemed to add new, great features in stable rust. Now? Crickets. Is frustration with Rust on the rise? I just started using Rust few month ago and absolutely love it. I can't tell what's going on with the Rust foundation so I can only judge by reading sentiments. Nothing would kill my…

>Nothing would kill my vibe harder than knowing smart people thinks the language isn't doing great :(

Smart people will always do that, I've found it's better to ignore the chatter and focus on your own experience.

Re: Rewriting Rust

#158

Post by josephg »

eterevsky wrote: post #106

I would gladly switch to a Rust fork without async. Even though this article is not about async per se, it’s clear that async makes most of the described problems worse.

Author here. I think async is a great feature to have - but I can't help but wonder if you're right. It might just be case of timing, but it seemed like once async was in the works, work on the rest of the language ceased. For awhile there all energy was poured into bikeshedding over how async would work. And I don't know if anyone is super happy with the result. Pin is a mess. Async functions still return anonymous objects - which cause all sorts of problems. And there still aren't coroutines in the language, even though async functions are implemented internally using coroutines anyway.

If we could go back in time and have the rust project decide to never implement async, I wonder what rust would look like today. There's a good chance the language & compiler would be much nicer as a result.

Re: Rewriting Rust

#159

Post by lifthrasiir »

josephg wrote: post #156
lifthrasiir wrote: post #129

Earlier quoted context omitted.

`if some_var.is_some_and(|x| some_expr)` is the current way to do that. It is less flexible and doesn't actually bind `x` into the conditional body (hence the proposal) but works today.

Yeah, but I almost always want to pattern match out the value when I do that. As I said in the post you can also write this: if let (Some(x), true) = (my_option, expr) { But then it doesn't short-circuit. (expr is evaluated in all cases, even when the optional is None). Both approaches are also weird . It'd be much better to just fix the language to make the obvious thing work.

I personally use two conditionals for such case. Yes, I might have written Rust too long to say this but it is more like a minor ergonomic fix and any solution should be generalizable into other use cases. The eventually accepted syntax, `if let PAT = EXPR {&& EXPR}` [1], is okay by itself but not (yet) generalized and that's my current complaint about it. The whole `{let PAT = EXPR &&} EXPR` should have been a valid expression in my opinion. (This exact suggestion is not in the RFC, the closest would be making `let PAT = EXPR` a boolean expression which really looks like a mistake.) Maybe I should check whether this was ever suggested...

[1] https://rust-lang.github.io/rfcs/2497-if-let-chains.html

Re: Rewriting Rust

#160

Post by josephg »

hnlmorg wrote: post #18
culebron21 wrote: post #8

I think adding per-crate permissions to do undoable/unsafe things will lead us to permissions hell of devops in big deployments. Like Amazon S3 with gazillion options. I think it's time to do something radically different with 3rd party deps. Even if we put aside safety issues, each crate brings ~10 more dependencies by default (i.e. without any features turned on), which bloats compile times. Maybe it's better to be…

That’s a solution people already use (vendering and / lock files) but it doesn’t solve this particular problem. The closest to a solution we have is dependency scanning against known CVEs. Having per-crate permissions is, I think, the only way languages can evolve past this hell hole we call supply chain attacks. It’s not a silver bullet, there will be edge cases that can be bypassed and new problems it creates. But…

Author here. Yeah thats my thinking.

I also think you probably only need to restrict your dependencies. If you have a dep tree like this:

    a
    |-b
      |-c
Then if crate a decides b isn't trusted, c would inherit the same trust rules. This would allow crates to be refactored, but keep the list of rules needed in big projects to a minimum. You just have to add explicit rules for sub-crates which need more permissions. Thats probably not a big deal in most cases.

(You might still, sometimes, want to be able to configure your project to allow privileged operations in c but not b. But thats an edge case. We'd just need to think through & add various options to Cargo.toml.)

Post reply on HN