Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Rust

josephg.com

Post reply on HN
Print view View original

61–70 of 410 posts

Re: Rewriting Rust

#61

Post by simonask »

armitron wrote: post #44
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

This is the main reason we have banned Rust across my Org. Every third party library needs to be audited before being introduced as a vendored dependency which is not easy to do with the bloated dependency chains that Cargo promotes.

I'm sorry, but that feels like an incredibly poorly informed decision.

One thing is to decide to vendor everything - that's your prerogative - but it's very likely that pulling everything in also pulls in tons of stuff that you aren't using, because recursively vendoring dependencies means you are also pulling in dev-dependencies, optional dependencies (including default-off features), and so on.

For the things you do use, is it the number of crates that is the problem, or the amount of code? Because if the alternative is to develop it in-house, then...

The alternative here is to include a lot of things in the standard library that doesn't belong there, because people seem to exclude standard libraries from their auditing, which is reasonable. Why is it not just as reasonable to exclude certain widespread ecosystem crates from auditing?

Re: Rewriting Rust

#62

Post by School-Cotton »

dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Why, concretely, does this matter?

Other than people who care about relatively obscure concerns like distro packaging, nobody is impeded in their work in any practical way by crates having a lot of transitive dependencies.

Re: Rewriting Rust

#63

Post by selfmodruntime »

alkonaut wrote: post #36
olivierduval wrote: post #34

> Most crates I use - like human-size or serde don't need any special capabilities to work. So we don't need to worry so much about their authors "turning evil" and adding malicious code to our software well... :-( Actually, it's obvious that some authors might "turn evil" dumbly, by abusing some kind of priviledged permissions. By chance, these kinds of supply-chain risks are "easily" identified because 1) the permi…

Is there a known ratio of crates that use unsafe to ones that don't? It feels like most nontrivial crates would often need some unsafe. But a system like this might create a scenario where crates offload some of their unsafe code into separate crates so they need updating less frequently (Much like the blah-sys versus blah crates).

> It feels like most nontrivial crates would often need some unsafe

As a frequent contributor to a number of crates, this isn‘t really true. Also, most popular crates actively deny use of unsafe.

Re: Rewriting Rust

#64

Post by skywal_l »

armitron wrote: post #44
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

This is the main reason we have banned Rust across my Org. Every third party library needs to be audited before being introduced as a vendored dependency which is not easy to do with the bloated dependency chains that Cargo promotes.

The dependency hell issue is not directly related to Rust. The Rust language can be used without using any dependency. Have you banned javascript and python too?

Re: Rewriting Rust

#65

Post by simonask »

gyre007 wrote: post #50

One of the things that hit me when I was picking up Rust was that I felt like it had every imaginable feature one could think of - I dont know if Rust team said no to anything (yes I know they obviously must’ve done) - and yet people wanted more and more (some justifiably, others less so) as the language “felt” incomplete or that the features thatd be used by 2% of devs are totally necessary in the language that is “…

I'm curious, what drama in the Rust community are you referring to?

I see some drama associated with Rust, but it's usually around people resisting its usage or adoption (the recent kerfuffle about Rust for Linux, for example), and not really that common within the community. But I could be missing something?

Zig is great, but it just isn't production ready.

Re: Rewriting Rust

#66

Post by saghm »

alkonaut wrote: post #36
olivierduval wrote: post #34

> Most crates I use - like human-size or serde don't need any special capabilities to work. So we don't need to worry so much about their authors "turning evil" and adding malicious code to our software well... :-( Actually, it's obvious that some authors might "turn evil" dumbly, by abusing some kind of priviledged permissions. By chance, these kinds of supply-chain risks are "easily" identified because 1) the permi…

Is there a known ratio of crates that use unsafe to ones that don't? It feels like most nontrivial crates would often need some unsafe. But a system like this might create a scenario where crates offload some of their unsafe code into separate crates so they need updating less frequently (Much like the blah-sys versus blah crates).

I suppose this depends on your definition of "nontrivial", but I don't think most would, unless you count the fact that some stuff in std is implemented with unsafe under the hood. The only times I've ever needed to use unsafe Rust code in 5~ years of writing it professionally was for interfacing with a vendor-specific C library, and that was only for the wrapper around it; the rest of the code didn't need to use unsafe.

Re: Rewriting Rust

#67

Post by raverbashing »

You know, I agree

And there's a lot of things that are weird or clunky

I honestly don't "get" the "no classes, just struct methods thing" and while, sure, C++ is kinda like that, but the ergonomics are weird. I'd much rather have the class/methods declaration as most languages do

Lifetimes are good but the implementation is meh. Most cases could do with a default lifetime.

Copy/borrow strictness is good to think about but in most cases we don't care? Copy should probably the default and then you borrow in special cases

Re: Rewriting Rust

#68

Post by JoshTriplett »

lifthrasiir wrote: post #53
dist1ll wrote: post #33

Earlier quoted context omitted.

You're right, the windows crate alone contributes 2.2M. I wonder if there's a way to deal with this issue.

The exact size of the `windows` crate depends on feature flags, because parsing 2.2M lines of code is always going to be very expensive even when you immediately discard them.

The parser is shockingly fast. The slow parts come after parsing, where we process all those function definitions and structure definitions, only to end up throwing 98% of them away.

A challenging architectural problem that several of us are trying to get someone nerdsniped into: inverting the dependency tree, such that you first check what symbols exist in a large crate like windows, then go to all the crates depending on it and see what they actually consume, then go back and only compile the bits needed for those symbols.

That'd be a massive improvement to compilation time, but it's a complicated change. You'd have to either do a two-pass compilation (first to get the symbol list, then again to compile the needed symbols) or leave that instance of the compiler running and feed the list of needed symbols back into it.

Re: Rewriting Rust

#69

Post by OtomotO »

armitron wrote: post #44
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

This is the main reason we have banned Rust across my Org. Every third party library needs to be audited before being introduced as a vendored dependency which is not easy to do with the bloated dependency chains that Cargo promotes.

Good on you, this approach will keep you employed for a looooooooong time, because someone has to write all that code then, right? ;)

Re: Rewriting Rust

#70

Post by jwr »

moss2 wrote: post #37
dist1ll wrote: post #15

I think the dependency situation is pretty rough, and very few folks want to admit it. An example I recently stumbled upon: the cargo-watch[0] crate. At its core its a pretty simple app. I watches for file changes, and re-runs the compiler. The implementation is less than 1000 lines of code. But what happens if I vendor the dependencies? It turns out, the deps add up to almost 4 million lines of Rust code, spread acr…

Same problem with JavaScript's NPM. And Python's PIP.

This isn't necessarily a language problem, though, more of a "culture" problem, I think.

I write in Clojure and I take great pains to avoid introducing dependencies. Contrary to the popular mantra, I will sometimes implement functionality instead of using a library, when the functionality is simple, or when the intersection area with the application is large (e.g. the library doesn't bring as many benefits as just using a "black box"). I will work to reduce my dependencies, and I will also carefully check if a library isn't just simple "glue code" (for example, for underlying Java functionality).

This approach can be used with any language, it just needs to be pervasive in the culture.

Post reply on HN