Live data from Hacker News

Regex: badly needs fuzzing

svn.boost.org

111–120 of 180 posts

Re: Regex: badly needs fuzzing

#111
post #96

Am i the only one, who get slowly angry about that "Use $language but Not this One"-comments? I don't see much value in such comments, srsly. Why don't accept the fact someone decided to write $it, and move on with usefull comments? Enough hn this today.

Yes, I had a similar thought.

The top two comment chains right now are (to paraphrase) "See, modern C++ isn't free of memory issues" and "Maybe we should rewrite it in Rust and compare".

Re: Regex: badly needs fuzzing

#112
post #2

Another counterexample to the idea that modern C++ written by experts is free of memory safety issues.

> modern C++ written by experts is free of memory safety issues

Which simply isn't good enough, it takes 10+ years to become expert level at anything, so what are people supposed to do to get to that level?

Re: Regex: badly needs fuzzing

#113
post #64

Practical upshot: do not feed untrusted regexes into boost::regex.

Untrusted regexes to anything , where possible. There are fairly well-known ways to use those to run a DoS: https://en.wikipedia.org/wiki/ReDoS

Note that it is possible to design regex engines that deliberately avoid these pitfalls (e.g. RE2, which was originally written for Google Code Search, as well as Go's regex engine (written by the RE2 author) and Rust's regex engine). The downside is that these engines are forced to disallow certain exploitable features, such as backtracking.

Re: Regex: badly needs fuzzing

#114
post #102

Earlier quoted context omitted.

> Well, yes if the strawman "all modern C++ written by experts is free from memory safety issues" is what you're countering. Well, I have seen exactly that sentiment. But, more importantly, this isn't exactly an obscure memory safety issue. It's a huge collection of flaws that showed up the instant Dmitry Vyukov threw a fuzzer at it. It's not just "all expertly-written C++ is free of memory safety issues" that this i…

> Well, I have seen exactly that sentiment. Then surely you can provide a reference to it. > it's also a counterexample to "most C++ written by experts doesn't have memory safety issues that matter in practice". This is "most C++ written by experts"? At least you're willing to back off your initial ridiculous assertion somewhat. This one isn't much better though. > I haven't brought up Rust here. Oh come on, you can'…

> This is "most C++ written by experts"?

Boost is peer-reviewed and receives more scrutiny than most C++ in the wild. Its peer review is its major selling point, in fact.

> Oh come on, you can't be serious given your advocacy for Rust.

I think it's possible for me to be able to express opinions in favor of memory safe programming languages in general (which is what I'm doing here) while also having worked on one.

If someone were to say "this illustrates why you should use Go for untrusted regexes", I'd agree with them!

Re: Regex: badly needs fuzzing

#115

Any rust lovers out there: Could I ask you do a benchmark comparison and a fuzz comparison. I'd be genuinely interested in the result and if (as you might hope) the Rust::regex is as fast as boost:regex, and never crashes, that would persuade at least me to finally learn some Rust!

It's not related to fuzzing but this article is definitely an interesting one that involved a lot of regex in rust http://blog.burntsushi.net/ripgrep/.

Re: Regex: badly needs fuzzing

#116

Earlier quoted context omitted.

Is there a quick and easy way to check if a particular regex could take exponential time?

All regexes run in O(N) where N is the length of the string matched. But some regex engines accept non-regular expressions. [0] The usual notation for it is an escaped number: \1 or \2 or so on. They're used to refer back to capturing groups earlier in the expression, usually marked by parentheses. Regular expressions don't have backreferences but various enhanced expressions add them. If you use those extensions, yo…

It's worth pointing out that if you're using a regex engine that only uses backtracking, then you can't assume all regular expressions take linear time. For example, running `(a)c` against `aaaaaaaaaa` takes exponential time in the number of `a` characters even though it is regular.

A hybrid regular expression engine could, in theory, recognize that a particular expression is regular and therefore use a finite state machine to guarantee linear time and space execution (where the size of the regex is held constant).

Re: Regex: badly needs fuzzing

#117
post #110
post #71

Earlier quoted context omitted.

pcwalton is not framing the problem in a particularly useful way. This is a question of risk management and his argument is basically that one should always reduce the risk of memory management errors to zero. Others say that they can tolerate some risk, as long as it's in acceptable margins, since it's expensive to totally eliminate it. I don't think that lecturing everyone "No, you really want to have 0 risk, you f…

> lecturing everyone "No, you really want to have 0 risk, you fools" On HN, please don't use quotation marks that make it look like you're quoting someone when you're not. It may seem a minor point, but we've found that it's important for clarity and respect. Also, "lecturing everyone" is borderline name-calling, which the site guidelines ask you not to do: https://news.ycombinator.com/newsguidelines.html .

Point taken about the quotes. It was thought as a figure of speech, but I can see how that could be misinterpreted.

"Lecturing everyone" is not at all name-calling. That's what he's been doing repeatedly, with code examples in several if not most threads related to C++ and safety.

I also don't consider pointing out to someone that they're lecturing offensive or abusive. In this case, this behaviour has derailed the thread and attracted criticism from many other people.

Re: Regex: badly needs fuzzing

#118
post #68

Earlier quoted context omitted.

Safety may seem like a binary property, but it's really not. Modern C++ is not as safe as Rust, but it is much safer than C, and significantly safer than doing manual memory management and raw pointer manipulation in C++. The interesting question is if that's enough for a particular project. In general, I would argue that it is, because security is but one of the non-functional properties of software and the types of…

> the types of bugs that Rust prevents compared to modern C++ are but one category of security-relevant bugs Memory safety bugs frequently result in basically the worst possible compromise imaginable: arbitrary remote code execution. It is possible to get RCE in other ways, but lack of memory safety makes it way way easier.

This doesn't change anything, you've just added another link at the end of the system > non-functional > security > code execution security chain.

Each project has the option at every link to decide that they're willing to accept a certain risk there and then tools delivering better results don't matter.

Re: Regex: badly needs fuzzing

#119

Earlier quoted context omitted.

Turning off your computer is the only way to avoid bugs. All software has bugs.

We aren't talking about eliminating all bugs. We're talking about eliminating memory safety problems, which frequently result in RCE.

What you were doing is advocating Rust in a C++ thread, again. Safety first, and all that. Congratulations on getting voted to the top of this topic, but it's tiresome.

Let's pretend C++ is a car. I get in my car, and I drive somewhere. Yes, there are hundreds of thousands of accidents per year, but really, most people get where they want to go, and we aren't all dead. I've had my share of fender benders, but no RCE has ever been exploited in 25 years of my C or C++ code.

Let's pretend Rust is a car. Half the time I want to go somewhere, I can't even get out of the driveway before I stumble into a limitation of the language or the standard libraries. One time it's overly strict coherence rules that forbid me from doing something that shouldn't break coherence [1]. Some other time it's because the standard library doesn't have a trait for a commonly implemented method, so I can't write a generic function to call that method. Yet another time, I find out you really can't accomplish the task without writing unsafe code, so the compiler really wasn't going to protect me from myself anyways [2].

So yeah, Rust is saving me from car accidents because I don't even make it to the road much less my destination half the time. Do you drive wearing a 5 point harness and a helmet? Do you really think a true expert couldn't make a safe C++ regex library?

--

[1] Honestly, I think you guys are screwed on that one. I suspect you're going to have to break backwards compatibility to really fix it, so I believe you'll leave it broken instead.

[2] Outside of FFI calls, I have no idea what rules I'm supposed to respect in an unsafe block. I guess I just can't drive my Rust car to that neighborhood then. Very safe indeed.

Re: Regex: badly needs fuzzing

#120
post #71

Earlier quoted context omitted.

pcwalton is not framing the problem in a particularly useful way. This is a question of risk management and his argument is basically that one should always reduce the risk of memory management errors to zero. Others say that they can tolerate some risk, as long as it's in acceptable margins, since it's expensive to totally eliminate it. I don't think that lecturing everyone "No, you really want to have 0 risk, you f…

Memory-safety bugs can lead to RCEs, which are considered unacceptable risks to most in computer science. Additionally, as Rust shows, memory-safety bugs can be checked by a computer, which can consistently apply those checks, making it an excellent bang-for-buck to use Rust or a comparable memory-safety checker. That's what pcwalton is saying.

How can you say these are considered unacceptable risks to most, when people write so much code in C and C++?

These risks are widely accepted and people are trying to mitigate them using various methods.

And that's what I'm trying to communicate: they're only unacceptable to pcwalton and the Rust community.

Post reply on HN