Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Rust

josephg.com

Post reply on HN
Print view View original

31–40 of 410 posts

Re: Rewriting Rust

#31

Post by xgb84j »

JoshTriplett wrote: post #26

> Now, there are issue threads like this, in which 25 smart, well meaning people spent 2 years and over 200 comments trying to figure out how to improve Mutex. And as far as I can tell, in the end they more or less gave up. The author of the linked comment did extensive analysis on the synchronization primitives in various languages, then rewrote Rust's synchronization primitives like Mutex and RwLock on every major…

What you describe is how development of basic packages that are part or on the level of the standard library should be done. The languages we are currently using will still be used decades from now. Slow good decisions now save much more time later on.

Re: Rewriting Rust

#32

Post by DanielHB »

dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

The fact is that dependency jungle is the prevalent way to get shit done these days. The best the runtime can do is embrace it, make it as performant and safe as possible and try to support minimum-dependency projects by having a broad std library.

Also I am no expert, but I think file-watchers are definitely not simple at all, especially if they are multi-platform.

Re: Rewriting Rust

#33

Post by dist1ll »

conradludgate wrote: post #29
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

I bet most of those lines are from the generated windows api crates. They are notoriously monstrous

You're right, the windows crate alone contributes 2.2M. I wonder if there's a way to deal with this issue.

Re: Rewriting Rust

#34

Post by olivierduval »

> Most crates I use - like human-size or serde don't need any special capabilities to work. So we don't need to worry so much about their authors "turning evil" and adding malicious code to our software

well... :-(

Actually, it's obvious that some authors might "turn evil" dumbly, by abusing some kind of priviledged permissions. By chance, these kinds of supply-chain risks are "easily" identified because

1) the permissions are an "easy" risk indicator, so you can priorize either to pin the version library (after validating it) or validate the new version

2) not so many libraries will use these permissions so you "have time" to focus on them

3) in these libraries, the permissions will tell you what system call/bad effects is possible, so will allow you to narrow even more the scope of investigation

So, IMHO, permissions are not really the end of all but only a tiny step.

The real problem is "how can human-size be used to subvert the program ?" For example: what is happening if the returned size "forget" or "add" 100 bytes to files bigger than 1 KB ? As a remininder, STUXNET was about some speed a tiny bit faster than planned and shown...

Re: Rewriting Rust

#35

Post by alkonaut »

> most uses of unsafe would also require explicit whitelisting.

I think this is probably where all proposed whitelist/capability proposal discussions end. It's going to be too many crates that are in that category for it to be useful.

A good first step (not sure if it's already taken tbh) would be to at least sandbox build execution. So that an attacker can't execute arbitrary code when your app is compiled.

Re: Rewriting Rust

#36

Post by alkonaut »

olivierduval wrote: post #34

> Most crates I use - like human-size or serde don't need any special capabilities to work. So we don't need to worry so much about their authors "turning evil" and adding malicious code to our software well... :-( Actually, it's obvious that some authors might "turn evil" dumbly, by abusing some kind of priviledged permissions. By chance, these kinds of supply-chain risks are "easily" identified because 1) the permi…

Is there a known ratio of crates that use unsafe to ones that don't? It feels like most nontrivial crates would often need some unsafe. But a system like this might create a scenario where crates offload some of their unsafe code into separate crates so they need updating less frequently (Much like the blah-sys versus blah crates).

Re: Rewriting Rust

#37

Post by moss2 »

dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Same problem with JavaScript's NPM. And Python's PIP.

Re: Rewriting Rust

#38

Post by dietr1ch »

Marking fixed stack size (and maybe even with an actual bound) would be helpful to ensure the tail-call optimisation is being done.

I don't think any language helps verifying that., and even in the ones that require it by spec, it's unclear if it's happening. Maybe you didn't really wrote a tail-recursive function because of a helper that you expected to be inlined. I guess it's easy to notice if you try to blow the stack in a unit test though.

Re: Rewriting Rust

#39

Post by Flex247A »

dist1ll wrote: post #33
conradludgate wrote: post #29

Earlier quoted context omitted.

I bet most of those lines are from the generated windows api crates. They are notoriously monstrous

You're right, the windows crate alone contributes 2.2M. I wonder if there's a way to deal with this issue.

Enabling FAT LTO reduces the final binary size but it isn't a permanent fix.

Re: Rewriting Rust

#40

Post by teaearlgraycold »

iforgotpassword wrote: post #22
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Maybe they can learn from the Javascript folks, I heard they're very good at this.

Not sure if you're serious and talking about tree-shaking - or joking and talking about left-pad.
Post reply on HN