This is your reminder to treat a regex with similar scrutiny as eval.
Regex: badly needs fuzzing
81–90 of 180 posts
Re: Regex: badly needs fuzzing
#82Earlier quoted context omitted.
> if you're allowing users to input arbitrary regex patterns you have a whole lot of other problems. Why? I am legitimately asking, you say that like it is meant to be obvious or common knowledge. But many regular expression engines are self contained entities that can crash but cannot expose information or poison other parts of the program. A DoS-like attack might be an argument against unfettered user inputed regul…
It would be easy to create a regex that consumes a lot of CPU power. It's very difficult to vet a regex in an automated fashion.
Then don't vet it. Just run it as is, and limit how many local resources it can consume (CPU cycles and RAM). Then add on a timeout for good measure, and you are good to go.
Re: Regex: badly needs fuzzing
#83Earlier quoted context omitted.
The position "modern C++ is safe and all C programmers are idiots" is repeated quite often here on HN. To be fair, it is always the same small group of people who do that.
Safety may seem like a binary property, but it's really not. Modern C++ is not as safe as Rust, but it is much safer than C, and significantly safer than doing manual memory management and raw pointer manipulation in C++. The interesting question is if that's enough for a particular project. In general, I would argue that it is, because security is but one of the non-functional properties of software and the types of…
> the types of bugs that Rust prevents compared to modern C++ are but one category of security-relevant bugs
Memory safety bugs frequently result in basically the worst possible compromise imaginable: arbitrary remote code execution. It is possible to get RCE in other ways, but lack of memory safety makes it way way easier.
Re: Regex: badly needs fuzzing
#84They seem to be fuzzing the regex, not just the input it is applied to. This may or may not change the results, but if you're allowing users to input arbitrary regex patterns you have a whole lot of other problems.
> if you're allowing users to input arbitrary regex patterns you have a whole lot of other problems. Why? I am legitimately asking, you say that like it is meant to be obvious or common knowledge. But many regular expression engines are self contained entities that can crash but cannot expose information or poison other parts of the program. A DoS-like attack might be an argument against unfettered user inputed regul…
Allowing users to enter regular expressions is definitely something that should only be entered into with caution. It is effectively code in a programming language that will be run, after all.
For example, it means their input needs parsing - and parsing is very hard to get right and frequently results in exploitable bugs. There are also potential problems where regexps could take a very long time to execute, which could be a denial of service issue.
Re: Regex: badly needs fuzzing
#85Earlier quoted context omitted.
I don't think it's ridiculous. It seems like almost every discussion about Rust vs C++ here has a few people saying that you don't need Rust's guarantees about memory safety if you're writing modern C++. And that oft-repeated comment is what pcwalton is referencing.
pcwalton is not framing the problem in a particularly useful way. This is a question of risk management and his argument is basically that one should always reduce the risk of memory management errors to zero. Others say that they can tolerate some risk, as long as it's in acceptable margins, since it's expensive to totally eliminate it. I don't think that lecturing everyone "No, you really want to have 0 risk, you f…
> This is a question of risk management and his argument is basically that one should always reduce the risk of memory management errors to zero.
No, that's not my argument.
> Others say that they can tolerate some risk, as long as it's in acceptable margins, since it's expensive to totally eliminate it.
These flaws are not "in acceptable margins". They were numerous and were found the instant Dmitry Vyukov turned a fuzzer on the library.
And it's not expensive to totally eliminate memory safety bugs. Just code in a memory-safe language. That's what most people already do.
Re: Regex: badly needs fuzzing
#86Earlier quoted context omitted.
It would be easy to create a regex that consumes a lot of CPU power. It's very difficult to vet a regex in an automated fashion.
Then don't vet it. Just run it as is, and limit how many local resources it can consume (CPU cycles and RAM). Then add on a timeout for good measure, and you are good to go.
Alternatively, where possible, use a regular expression engine that does not have that issue: https://swtch.com/~rsc/regexp/regexp1.html
Re: Regex: badly needs fuzzing
#87Earlier quoted context omitted.
From my understanding the main benefit of RE2 is not speed, but linearly scaling execution time with respect to the input size, along with bounded memory usage. For certain inputs, it may outperform other engines, but the converse may also be true. As with any feature that may be abused, backtracking may also be useful: for example, you may need to write a script to munge text. Since you're not exposing it to arbitra…
RE2 is not dramatically faster than all other regex implementations, but it is dramatically faster than boost::regex, which is among the slowest I've ever tested.
Ah, fair enough.
Re: Regex: badly needs fuzzing
#88Another counterexample to the idea that modern C++ written by experts is free of memory safety issues.
Well, yes if the strawman "all modern C++ written by experts is free from memory safety issues" is what you're countering. I find that to be gratuitous and petty, and not a good representation of Rust, however.
> Well, yes if the strawman "all modern C++ written by experts is free from memory safety issues" is what you're countering.
Well, I have seen exactly that sentiment. But, more importantly, this isn't exactly an obscure memory safety issue. It's a huge collection of flaws that showed up the instant Dmitry Vyukov threw a fuzzer at it. It's not just "all expertly-written C++ is free of memory safety issues" that this is a counterexample to: it's also a counterexample to "most C++ written by experts doesn't have memory safety issues that matter in practice".
> I find that to be gratuitous and petty, and not a good representation of Rust, however.
I haven't brought up Rust here. You can eliminate the memory safety issues by writing in Go, or Java, or C#, etc. etc.
Re: Regex: badly needs fuzzing
#89Another counterexample to the idea that modern C++ written by experts is free of memory safety issues.
No, but not everything needs to be memory safe. That's what the kernel is there for. You can typically just restart the program. The classic program safety vs programmer time tradeoff.
> No, but not everything needs to be memory safe. That's what the kernel is there for.
The kernel doesn't automatically defend against RCE.
Re: Regex: badly needs fuzzing
#90Earlier quoted context omitted.
Avoiding bugs is a pretty effective way to not have bugs.
Turning off your computer is the only way to avoid bugs. All software has bugs.
We aren't talking about eliminating all bugs. We're talking about eliminating memory safety problems, which frequently result in RCE.