Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malicious Rust crate Arrayref runs a build-time payload

safedep.io

Post reply on HN
Print view View original

341–350 of 529 posts

Re: Malicious Rust crate Arrayref runs a build-time payload

#341

Post by josephg »

leecommamichael wrote: post #171
Ygg2 wrote: post #169

Earlier quoted context omitted.

Spoiler alert! JavaScript has no language provided package manager. If Odin gets moderately successful someone will probably reinvent it.

NPM came along in 2010, Javascript was huge before that. It was only when people wanted a common approach to shipping both in browser and "native" that this glitch happened. I believe a standard library and "batteries" would have abated it entirely or resulted in a slightly less-bad situation. I do think Cargo is a slightly less-bad situation in many ways, and that it can be done better.

> It was only when people wanted a common approach to shipping both in browser and "native" that this glitch happened.

Nah. Npm was invented because node had its node_modules directory. But it was tricky to find and download modules you wanted to use. Until npm, you had to add libraries to node_modules by hand. And check them in to git or something. And keep them up to date somehow. Npm added a searchable index and a tool to automatically install all your modules. Npm was only bundled alongside Nodejs many years later.

Bundling was separate. I can’t remember if browserify predated npm or not. But it was a wild idea at the time to make node modules build for the browser too. Browserify - and later webpack and friends - work with or without npm.

Re: Malicious Rust crate Arrayref runs a build-time payload

#342

Post by VorpalWay »

cosmic_cheese wrote: post #78

I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable. I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 tot…

> I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.

Nobody can agree on what those common things are in a general purpose langusge though. It works for something like Go, because it is largely used for servers and command line tools from what I can tell.

For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.

And even if did put some of those things in std, that is stable forever. That is how C++ ended up with an unusably slow regex implementation in the standard library. And it isn't fixable. Everyone just ignores it and uses a third party library like PCRE, RE2, etc instead (depending on their specific needs). I would not expect web or GUI things in the standard library for this reason, those standards change too frequently for that to be stable forever.

And even something like Python, which is known for data science and machine learning, doesn't have core libraries for those activities in the standard library. Numpy, Pandas, Scipy, Pytorch etc are all third party projects.

Re: Malicious Rust crate Arrayref runs a build-time payload

#343

Post by VorpalWay »

josephg wrote: post #322
Pannoniae wrote: post #291

Earlier quoted context omitted.

The fallacy is assuming that standard != unchangeable. One can very simply....make breaking changes. Yes it will require some work to update but that's already the case when you upgrade library versions. The dependency hell doesn't help anyone here, the downside is much worse - lots of bloat, overgeneric libraries and awful supplychain security.

I’m a big fan of how C# handles this. The standard library is versioned like any other library. Your program is explicit about the major version of the standard library you’re pulling in. And all major versions of the standard library continue to work. For rust, this would mean something like adding std = 1.0.6 to your cargo.toml. The advantage is it means the standard library can deprecate and replace features. Upgr…

> The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.

C++ went through that around C++11, where some types had to have ABI breaks (on some implementations, such as that of GCC's libstdc++). It has been a while, but as I remember it std::string was affected since it went from COW to SSO. Map might have been affected too (don't quite remember). It was a mess.

Rust can link multiple semver versions of the same dependency, but you can't pass data structures between those versions. For transitive dependencies that usually isn't a big deal if your direct dependencies use them internally rather than exposing the types in their own APIs. But std contain vocabulary types that everyone uses (Option, Result, String, etc) so that would really not work.

Re: Malicious Rust crate Arrayref runs a build-time payload

#344

Post by josephg »

cosmic_cheese wrote: post #247
TZubiri wrote: post #235

Earlier quoted context omitted.

>I think we should be taking a more “batteries included” approach https://www.youtube.com/watch?v=GZOuz-SG7-g Funny how you skipped "I should build my own batteries" and when straight to "increasing and centralizing the duties of your main gratis 'vendor'".

I don't like the "bring your own" approach because unless one happens to be an absolute tour de force 10x engineer unstoppable god of a programmer (which most of us, myself included, are not), whatever you build is never going to be as well-rounded, fleshed out, and complete as something built by a larger organization, especially when it comes to UI libraries (which are monstrous projects if done right, e.g. meeting…

The counterpoint is that whatever std invents is probably not going to be as good as what moviated people in the community make.

The rand crate is my favorite example of this. They have a whole bunch of different rngs. All rated by quality and performance. Some are csrngs and some aren’t. It’s a delight. “Batteries included” languages don’t come close.

Then there’s serde - which uses a clever technique with traits to allow compile time specialisation and optimisation of binary and json serialisers. Most people had no idea that was even possible in rust until someone in the community made it.

Maybe std should pick up the best crates from the community and bring them in house? This might be a good idea. But also maybe not. Since serde came out, several other crates have found ways to crush its performance numbers. Often by 2-5x if memory serves. What a relief we didn’t immortalise the slow version of serde, right?

I’m playing devils advocate here. I think std should be a bit bigger too. But there are real tradeoffs in doing so.

Re: Malicious Rust crate Arrayref runs a build-time payload

#345

Post by VorpalWay »

cosmic_cheese wrote: post #209
mightyham wrote: post #191

Earlier quoted context omitted.

Rust has a sizable and well featured standard library at this point. I think it would be absurd to claim that the base language is "near-unusable" if you are using it for system programming, which is its intended use case. Huge standard libraries make sense for Java, Go, Apple platform etc. because they are developed by giant enterprises that can manage the overhead. Thinner modular systems are a more natural fit for…

It’s being used for a wide variety of other purposes however, and so perhaps it’s time to adapt to that reality. While overhead is a real concern, I think it’s often blown out of proportion. Once a language achieves a certain baseline of stability and isn’t in constant flux and the standard library matures, changes become infrequent and maintenance load is low. The work is heavily front-loaded.

Not at all true, look at the large parts of Python's standard library that are effectively unusable or irrelevant. And they even allow some breaking changes over time (unlike Rust). This is especially true when it comes to anything related to internet protocols or file formats, but there are other modules that have far better replacements on PyPi too (re vs regex for example).

Re: Malicious Rust crate Arrayref runs a build-time payload

#346

Post by eminence32 »

cogman10 wrote: post #281
eminence32 wrote: post #203

Earlier quoted context omitted.

> We shouldn't need to resort to pasting `find` commands [1] into the shell from blog posts to tell if we're compromised. I know you recommended `cargo audit`, but that is not nearly as universally installed as `find`. And as far as I know, cargo-audit has to be run within a project directory? Can it be run outside of the context of a project and scan the cargo registry cache? I personally appreciated having the `fin…

I believe what's being suggested is that `cargo audit` should be a part of cargo rather than an add on. You'd obviously already have cargo installed, which means you should be able to run a command against cargo to see if you are currently exposed to any security problems. Even if it's not the current "audit" add on.

Well, but as a system admin, I might not have cargo installed even if my users have it installed in their home directory. So a simple non-Cargo command (like find) has a lot of advantages for me

Re: Malicious Rust crate Arrayref runs a build-time payload

#347

Post by cloudfudge »

segphault wrote: post #195
cosmic_cheese wrote: post #78

I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable. I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 tot…

This is spot on. I am extremely uncomfortable with the large number of transitive dependencies that end up in pretty much any non-trivial rust application. No amount of memory safety will save us if a tiny, ubiquitous library that nobody scrutinizes because it’s nested eight layers down the dependency graph gets compromised. I think Go apps tend to have better dependency hygiene because the language has a better stan…

I think one thing that has made Go have better dependency hygiene is not merely having "batteries included" in the standard lib, but the early focus on having production-ready implementations of a lot of stuff in the standard lib, as opposed to minimum viable implementations. I used to get a long way with one or two dependencies that would barely fan out at all. It's been a while since I worked on a Go project, so I'm not sure if it's still that way.

Re: Malicious Rust crate Arrayref runs a build-time payload

#348

Post by josephg »

7373737373 wrote: post #253
sempron64 wrote: post #246

Earlier quoted context omitted.

Sandboxing is a mitigation but a very limited one. The library itself can contain a malicious payload -- rust code most often ends up as native code executed on the host. I think we may need to enter a world where there are fewer, heavily audited, libraries and dependency depth is limited. Allowing unvetted dependencies to be installed by default is not a good way forward. App stores have a similar problem and I thin…

That's why languages need sandboxing at runtime as well

Yes, or the compile time equivalents. Rust’s safety gets you most of the way there. We just need a capability model in the language, a more limited std and a way to ban untrusted 3rd party libraries from using unsafe code without explicit permission. I dream of a world where a function with the signature of add(u32, u32) -> u32 can’t burn my house down and steal my wife.

Functions should only have access to their arguments. Nothing more. We need to end ambient authority.

Re: Malicious Rust crate Arrayref runs a build-time payload

#349

Post by the8472 »

Pannoniae wrote: post #291
echelon wrote: post #223

Earlier quoted context omitted.

Python is batteries included, but all the batteries have corroded. Look at C++'s long in the tooth STL. You don't want to marry a language to fast aging libraries you have to support for eternity. Better libraries always naturally emerge. Rust's decision here is fine. The only thing I'd like to see is the ability to programmatically limit transitive dependency count or depth in Cargo. I'd also like crates to specify…

The fallacy is assuming that standard != unchangeable. One can very simply....make breaking changes. Yes it will require some work to update but that's already the case when you upgrade library versions. The dependency hell doesn't help anyone here, the downside is much worse - lots of bloat, overgeneric libraries and awful supplychain security.

Rust releases a new version every 6 weeks and the latest version is the only supported version. This is only tenable when the amount of breaking changes is tiny and each instance's impact is analyzed and cost-benefit tradeoff is checked and mitigations are put in where possible.

Some features, e.g. the never type[0], are held back years due to all the work that's needed to minimize the breakage.

[0] https://youtu.be/3jM4cnEVrLc?t=271

Post reply on HN