Afl.rs: Fuzzing Rust code with american-fuzzy-lop

Correct is relative to the goals of the person assessing the comparison. If that person values performance over safety to enough of a degree, them a small performance loss will outweigh safety (regardless of whether you or I think this is a useful way to compare). If your goal is get people using Rust, and you believe people value performance, you can't ignore this and expect things to turn out how you like. This is the balancing act of a community and of advocates, making hard choices about what they would like to do, and what they feel is needed due to the constraints of reality.

> The same can be said of the performance comparison of "array index bounds checking". One program works, the other does not, so it isn't a fair comparison.

No, for some subset of array bounds checking, one works and the other doesn't. For the rest, they both work, because it isn't out of bounds. Whether you think people should program defensively and assume their code could fail, some will instead choose to assume they can deal with that as the programmer and choose to do so because it is more performant. You can either write those people off, or meet them half-way, and hope to escort them the rest of the way. Rust apparently chose the latter.

yep. I was referring to "If not via rustup, I would expect everything that can be fetched via rustup to be in the package repo."

What I meant was: instead of users installing rustup and then installing all the toolchains through rustup (`rustup install stable`, `rustup install arm-linux-gnueabi`, `rustup install clippy`), they can also install `rustc`, `rustc-arm-linux-androideabi` (cross toolchain), `rust-clippy` through the package manager (apt-get install or whatever). Cross compile toolchains for C are often similarly packaged (gcc-cross, etc).

The way I understand it (again, unofficial, not sure if it's the actual plan :) ), we would like most users to use rustup as the single source of truth for Rust. This won't work for some setups, including distro packaging, and I'm not sure what the plan is there, but I'd expect it to be something similar to what I wrote above. The binaries that rustup downloads will always be available, regardless, if you are okay with downloading a signed binary once and internally distributing it. But I don't expect it will come to that. Though right now Rust hasn't yet gotten all the ducks in a row for proper distro packaging, so that's all you can use. This is being actively worked on, and IIRC all the major work is done (?)

Having any output that's based on a person remembering to set a configuration is less useful in these situations than having the configuration hard coded. (you don't want someone hunting down the correct config in some company wiki, much less working from memory. At most you want a config copied from a repository of configs).

Having a utility download a binary or source from the internet is less useful in these situations than serving it locally. (You can't control that the remote resource is still there, is the same, is secure).

Knowing that the exact same stuff that works in whatever development environment(s) you have (local, shared, both) works the same as when it's pushed to some further resource (automated build server, automated test harness, etc) is important in preventing bugs and problems.

In the environment I had experience with, we had a mainly Perl ecosystem running on CentOS servers. We created RPM packages for every perl module we pulled in as a dependency for our system or back-end RPC servers. Everything we installed on these boxes got an RPM built if at all possible. We maintained our own yum repository that we installed these packages from. While trivial to run cpan to install a module on these systems, that was not deemed acceptable for anything going into production. Rustup would not have been allowed into production here, nor on some of the shared dev and testing boxes we had, since that wouldn't lead to being able to build the exact same binaries easily and definitively. The absolute last think you want is a problem that loses data/config, and to find that you're not sure how to reproduce the last build environment exactly.

Does rust make assurance that things should build correctly with later versions? Yes. Does that really matter when you're talking about hundreds of thousands to millions of dollars? Not without a contract. That's one reason why enterprises use distributions that provide this feature, and build their own packages and deploy them where those distributions fall short.

I already addressed that; I'm using rustup as a toolchain manager, not for downloading. You can have a .multirust directory that everyone uses and use rustup to switch. You could alternatively set rustup to download from a internal server that contains a reduced set of preapproved binaries only. I'm assuming that the external network is turned off here; it usually is in these situations. If you want a tool that manages toolchains but does not contain code that accesses the internet, that's a more stringent requirement that rustup doesn't satisfy; though I do hope Rust's distro packages work with update-alternatives.

You can also create an rpm for your specific two-compiler setup, of course. That's annoying though.

> Having any output that's based on a person remembering to set a configuration is less useful in these situations than having the configuration hard coded

Set default to stable, and `alias fuzz='rustup run nightly cargo afl'` :)

Again, this is for tooling, you can easily paper over the fact that the tool uses a different compiler.

Rust does have a tool (crater) that runs releases on the entire public ecosystem before releasing. There is an intention to make it possible for private code to take part in this too and provide feedback. But you're right, it is still unreasonable to expect enterprise users to trust that updates will work :) Any stability guarantee isn't enough, since bugs happen.