Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Afl.rs: Fuzzing Rust code with american-fuzzy-lop

github.com

Post reply on HN
Print view View original

81–90 of 110 posts

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#81

Post by kbenson »

Manishearth wrote: post #75
yazaddaruvala wrote: post #67

Earlier quoted context omitted.

Lol, you're joking right? With respect, the Rust community is extremely in-exposed to the majority of actual enterprise environments, and that's ok. Honestly the best path forward is to just acknowledge it, say "we are working on it" and you'll get your point across better. Again with respect, because you all do really amazing work, fighting it honestly makes you and the whole community come across ignorant.

I'm not joking. Did you miss the part where I said the situation is worse for gcc? I do understand that this is a problem. I also do understand that it is not specific to Rust. Besides, convincing IT to allow rustup with an access-controlled toolchain directory shouldn't be a big deal in many cases. If they allowed one compiler, why not two? (Rustup can also be installed without sudo/admin, but I assume you are talki…

> The general evolution of Rust is towards everything being done through rustup.

The general evolution of Perl was towards everything being done through cpan (the client). Decades ago. That doesn't stop companies from rolling their own Perl packages (when not just using what the system shipped with) and CPAN module packages (when it doesn't already exist as a package for the distro, or in a trusted third party repo, or is not the right version your code needs).

In many enterprise environments, when one or more production servers running your code fails, you want clueless intern #23 to be able to spin up a new box, running the default configuration for your distro, and pop the required packages on for your program, copy the relevant config or two that are needed for it to event start, start it up and expect it to "just work". Running rustup and remembering it needs to install a specific version to ensure everything turns out right is not conducive to that.

Now, this is less of a problem since rust is used to build other stand-alone programs, not necessarily run itself on as many servers, but it we're talking about a QA or testing box that dies, and there's a critical fix waiting to go out, I think it's obvious where this still has some importance.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#82

Post by Manishearth »

kbenson wrote: post #81
Manishearth wrote: post #75

Earlier quoted context omitted.

I'm not joking. Did you miss the part where I said the situation is worse for gcc? I do understand that this is a problem. I also do understand that it is not specific to Rust. Besides, convincing IT to allow rustup with an access-controlled toolchain directory shouldn't be a big deal in many cases. If they allowed one compiler, why not two? (Rustup can also be installed without sudo/admin, but I assume you are talki…

> The general evolution of Rust is towards everything being done through rustup. The general evolution of Perl was towards everything being done through cpan (the client). Decades ago. That doesn't stop companies from rolling their own Perl packages (when not just using what the system shipped with) and CPAN module packages (when it doesn't already exist as a package for the distro, or in a trusted third party repo,…

This is local developer tooling we're talking about, however. It doesn't need to run on those machines.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#83

Post by kbenson »

Manishearth wrote: post #82
kbenson wrote: post #81

Earlier quoted context omitted.

> The general evolution of Rust is towards everything being done through rustup. The general evolution of Perl was towards everything being done through cpan (the client). Decades ago. That doesn't stop companies from rolling their own Perl packages (when not just using what the system shipped with) and CPAN module packages (when it doesn't already exist as a package for the distro, or in a trusted third party repo,…

This is local developer tooling we're talking about, however. It doesn't need to run on those machines.

You want your devs all running the exact same version of the compiler. The last thing you want is someone with a slightly different version than used by everyone else (and production!) which writes some code that compiles and runs fine without problems, but crashes or exhibits a bug in testing, QA, or worst case, production. Having something like that occasionally gum up your development pipeline is a good reason for pegging tool versions (depending on the type of tool).

Having things use rustup is great. Don't assume that's sufficient for certain populations of users (and a populations you really may want, such as enterprise). At best, they'll use rustup as a basis to start some other packaging, but even that's risky because rustup is complex, and it's hard to contain all the variables unless you're very careful about exactly what version you use with it, so then you're packaging rustup, and then package what it created. Easier at that point to just skip rustup.

It's all about reproducibility, which is king in enterprise. Rustup is about ease of use, and easy iteration. Enterprise already sacrifice that on the alter of reproducibility, so it's really not a selling point for them. It's great for everyone else, it just has less to offer for this subgroup.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#84

Post by Manishearth »

kbenson wrote: post #80
Manishearth wrote: post #78

Earlier quoted context omitted.

In rusts case it sounds like we will ship rustup.rs in distros, since it is also intended to be how people obtain cross compilers and whatnot. I could be wrong about this -- I am aware of, but not directly involved in, the plans for rustup. If not via rustup, I would expect everything that can be fetched via rustup to be in the package repo. This solves the problem for clippy but not for afl, though there are solutio…

> In rusts case it sounds like we will ship rustup.rs in distros I sincerely doubt any enterprise level distribution will make that the primary source of obtaining a rust compiler. I would bet good money that Red Hat won't. It's counter to most of the reason enterprise distributions exist in the first place, which is a stable (often at the cost of many other feature) base on which to build and support. If they aren't…

Right, which is why I said for such distros I expect rustup-obtainable extras to be packaged as well and installed in the same way. Again, unsure if this is the actual plan.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#85

Post by Manishearth »

kbenson wrote: post #83
Manishearth wrote: post #82

Earlier quoted context omitted.

This is local developer tooling we're talking about, however. It doesn't need to run on those machines.

You want your devs all running the exact same version of the compiler. The last thing you want is someone with a slightly different version than used by everyone else (and production!) which writes some code that compiles and runs fine without problems, but crashes or exhibits a bug in testing, QA, or worst case, production. Having something like that occasionally gum up your development pipeline is a good reason for…

Right, you can pin to a nightly for tooling.

The solution I proposed does not affect reproducability, since you would only use the nightly when running local tools.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#86

Post by [deleted] »

Manishearth wrote: post #34
pjmlp wrote: post #29

Earlier quoted context omitted.

Every time someone posts a cool Rust library, most of the time it makes use of nightly. How come can I advocate Rust to serious enterprise customers, if I have to justify using nightlies to make use of such tooling? Apparently downvoting every time someone mentions the issue with nightly being required is fair.

> Every time someone posts a cool Rust library, most of the time it makes use of nightly. Examples? Examples that aren't tools, that is. The main example I can think of is serde, and that works on stable (just works better on nightly). What's wrong with nightly for using tooling? You don't have to keep updating, you can pin to a nightly from a particular date. You can use rustup.rs to switch to stable locally if you…

[deleted]

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#87

Post by pslam »

kbenson wrote: post #69
pslam wrote: post #68

Earlier quoted context omitted.

> if the performance cost can be made low enough I think that's the wrong way to look at it. How about instead: "Checks can be disabled if the safety cost is worth the performance increase" The "performance case" is a false baseline. You never had that performance in the first place, because your code didn't work.

It's a fine line to walk. Performance is a deal breaker for a lot of people, whether it rightly should be of not. It doesn't matter how much safer Rust is if it's not used much. If your goal is purely to make Rust as safe as possible, then you should advocate the view in your comment. If your goal is to increase Rust adoption, and/or decrease the use of less safe languages for development in the future (in whichever…

> If your goal is purely to make Rust as safe as possible, then you should advocate the view in your comment.

> If your goal is to increase Rust adoption, and/or decrease the use of less safe languages for development in the future (in whichever prioritization of the two you like), then IMO the route described in ekidd's comment seems the quickest route there, IMO.

My view is this is a false dilemma.

The "performance" case is not performance, because the benchmark is broken code. It does not work, because it is not secure/safe/reliable. It is incorrect to compare the performance of code compiled without overflow checking against code which is compiled with it enabled. They simply aren't the same program.

A correct performance comparison would be a program with manually inserted overflow checks everywhere, against a program with the overflow checks done automatically (by compiler).

Or, a performance comparison where code is selectively opted-out, where a critical path bottleneck is identified.

The same can be said of the performance comparison of "array index bounds checking". One program works, the other does not, so it isn't a fair comparison.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#88

Post by kbenson »

Manishearth wrote: post #84
kbenson wrote: post #80

Earlier quoted context omitted.

> In rusts case it sounds like we will ship rustup.rs in distros I sincerely doubt any enterprise level distribution will make that the primary source of obtaining a rust compiler. I would bet good money that Red Hat won't. It's counter to most of the reason enterprise distributions exist in the first place, which is a stable (often at the cost of many other feature) base on which to build and support. If they aren't…

Right, which is why I said for such distros I expect rustup-obtainable extras to be packaged as well and installed in the same way. Again, unsure if this is the actual plan.

I'm not sure where you said that, so I'm not sure what context to interpret that in. Does "rustup-obtainable extras to be packaged as well and installed in the same way" mean packaged through distro specific packaging mechanism, or something else?

My only point in this part of the thread, which has little to do with the "we use gcc because it's stable" argument, is just that you shouldn't expect the distros the change how they ship just for rust, which expecting them to use rustup would be (in what I think are the majority of cases). If rust as a community expects rustup to be what distros use, then I think there's a disconnect from reality happening, and it's worth looking at because if that's important to rust (which I don't know, honestly), then it should be addressed. I understand that you may not know if this is the actual plan.

Note: I don't particularly think rust requirly a nightly build for this is important. Organizations will look at it and assess it as a tool as needed, and if they think it's worth using and they want to roll it out, they'll package it in some manner if that's important to them (and that might just be a tarball, at the simpler level), or they won't if it's not. Stability is important, but at the tooling level, it's much less so, and being able to make sure all yours devs are in lockstep is probably more important (for large orgs).

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#89

Post by kbenson »

Manishearth wrote: post #85
kbenson wrote: post #83

Earlier quoted context omitted.

You want your devs all running the exact same version of the compiler. The last thing you want is someone with a slightly different version than used by everyone else (and production!) which writes some code that compiles and runs fine without problems, but crashes or exhibits a bug in testing, QA, or worst case, production. Having something like that occasionally gum up your development pipeline is a good reason for…

Right, you can pin to a nightly for tooling. The solution I proposed does not affect reproducability, since you would only use the nightly when running local tools.

My point is that it's not always a local tool. Sometimes what you consider a local tool might be something run on a QA or testing box, which needs to be tightly controlled to make sure that the difference between a tooling item intalled and a required library that is in production is closely tracked, and the tooling library needs to match exactly what's actually installed locally for devs. Think CI tool chains, automated testing, etc. A certain percentage of organizations where see "let's use this program that dynamically downloads and builds a program, which we can ask for a specific version if we want" as unacceptable because either it requires remote sources, it requires specific configuration which isn't tracked automatically or enforced, or both.

It's not about whether it's possible, it's about whether you're meeting the needs of the target audience. I don't believe rustup is meeting those needs in some cases. The important distinction is that possible does not imply acceptable.

Re: Afl.rs: Fuzzing Rust code with american-fuzzy-lop

#90

Post by yazaddaruvala »

Manishearth wrote: post #75
yazaddaruvala wrote: post #67

Earlier quoted context omitted.

Lol, you're joking right? With respect, the Rust community is extremely in-exposed to the majority of actual enterprise environments, and that's ok. Honestly the best path forward is to just acknowledge it, say "we are working on it" and you'll get your point across better. Again with respect, because you all do really amazing work, fighting it honestly makes you and the whole community come across ignorant.

I'm not joking. Did you miss the part where I said the situation is worse for gcc? I do understand that this is a problem. I also do understand that it is not specific to Rust. Besides, convincing IT to allow rustup with an access-controlled toolchain directory shouldn't be a big deal in many cases. If they allowed one compiler, why not two? (Rustup can also be installed without sudo/admin, but I assume you are talki…

First, I do want to say, I don't really care about the gcc alternative. I'm not even particularly interested in this tool. I've just found in the past, folks in the Rust community are quite dismissive about "other perspectives", particularly when they differ from "the open development way". I was just trying to urge you not to encourage that dismissive attitude.

Meanwhile, to potentially help you understand how, at least the company I work at functions:

Any time I need a library or tool, I need to:

1. Get approval from the open source team (for each minor version).

2. Start a security approval process (for each minor version).

3. Download the source, successfully modify the build scripts to use the internal build system and dependency management solution (for each minor version).

4. Upload the modified source to the internal dependency management solution (for each minor version).

5. Finish the security approval: Hopefully with a "Successful" ending (for each minor version).

Something like Rustup, in its current state, will not be usable (also its a bit redundant - we pin compiler/runtime versions in our BuildSystem.config). FYI, even a version of Cargo that can pull dependencies from a non-controlled repo, will not get approved (we would need to modify it to panic instead of call the network). Only a modified version which allows the internal build system to pull libraries from our internal dependency management solution, put the files in the correct dirs, will be allowed (fortunately I think Cargo is already able to do this). FWIW, this process would be almost identical to what we currently have to do for NodeJS projects and NPM packages.

Hopefully this gives more insight into why I thought it was funny, because:

1. Literally any "uncontrolled" tool that can talk to the network, download/build an executable, is entirely disallowed.

2. Getting approval for a single minor version is a pain. Requesting approval for a "nightly" i.e. unversioned is unimaginable. This would literally be the conversation:

What is the level of support on that version? Well, its not really versioned, so there is no level of support apart from upgrading to the latest nightly.

How would you update for a security fix? Update to the latest nightly.

Wouldn't that be an arbitrary number of commits, not just the security update, i.e. potentially the difference between minor/major versions? Yes.

So, we cannot minimally update the nightly for a security fix unless we cherry-pick the commit ourselves? Correct.

Also there is no support on the version of nightly we are pinned to? No.

So there may not even be a commit we can cherry-pick, we might have to investigate and make the change ourselves? Correct.

Also, there could potentially be a bug in our pinned version which only our company is exposed to? Well, its unlikely any other team/company will pin to that exact version, so yes, its unlikely we would even find out about a potential security vulnerability.

... lets wait for a version with more support.

Post reply on HN