Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we applied fuzzing techniques to cURL

blog.trailofbits.com

Post reply on HN
Print view View original

31–40 of 84 posts

Re: How we applied fuzzing techniques to cURL

#31

Post by burnished »

SamuelAdams wrote: post #10

I am curious how much effort goes into creating and maintaining unit tests and fuzzing tests. Sometimes it takes longer / more lines of code to write thorough tests than it does to implement the core feature. At that point, is it worth the time invested? Every new feature can take 2-3 times longer to deliver due to adding tests.

Generally speaking, yes. Its not like most code isn't changed as a consequence of writing those tests, so the practice has immediate benefit, but future changes can be made swiftly and securely due to the confidence those tests should be giving you.

But also your time estimate does sound wonky, 2-3x sounds extreme. Maybe you need to improve your test writing process?

Re: How we applied fuzzing techniques to cURL

#32

Post by stefan_ »

PoignardAzur wrote: post #13

I don't get the part about custom mutators: > If the data can’t be parsed into a valid TLV, instead of throwing it away, return a syntactically correct dummy TLV. This can be anything, as long as it can be successfully unpacked. If you're creating a dummy value, how is that better than failing? How does that give your fuzzer better coverage?

Remember TLV is for "Type Length Value". It's better to take your fuzzer output for value (or possibly type and value) but generate the length and final TLV yourself than having tons of fuzzer generated sequences already fail at the very basic type/length check that is unlikely to be vulnerable in software like cURL (but can be in many others..).

Re: How we applied fuzzing techniques to cURL

#33

Post by tgv »

ape4 wrote: post #9
tgv wrote: post #6

Earlier quoted context omitted.

Indeed. It's more of an argument not to execute scripts downloaded from the web. Which is valid, but not the point of fortifying curl.

Would it make sense for cURL to detect when its being piped into `sh`? Possibly have it refuse to proceed unless the site is on a whitelist or the user has added an --i-accept-the-risk option.

That's not possible in general, I think. Can you even get process information out of file descriptors? But the people who write the scripts that embed such curl commands, will add the option directly. And others will put it in the command to copy. Or put | cat | sh in the command. And there's also wget.

I think it's not something curl can tackle by itself. Instead, you'd need a file descriptor that informs you about the source, and the shell should refuse not whitelisted sources.

Re: How we applied fuzzing techniques to cURL

#34

Post by ape4 »

tgv wrote: post #33
ape4 wrote: post #9

Earlier quoted context omitted.

Would it make sense for cURL to detect when its being piped into `sh`? Possibly have it refuse to proceed unless the site is on a whitelist or the user has added an --i-accept-the-risk option.

That's not possible in general, I think. Can you even get process information out of file descriptors? But the people who write the scripts that embed such curl commands, will add the option directly. And others will put it in the command to copy. Or put | cat | sh in the command. And there's also wget. I think it's not something curl can tackle by itself. Instead, you'd need a file descriptor that informs you about…

You're right it subverted in general. isatty() can tell if output is being piped. When piped special safety rules could kick in. Of course they would be false positives but then there could be an option (like I mentioned above).

Re: How we applied fuzzing techniques to cURL

#40

Post by tonyarkles »

gavinhoward wrote: post #30
guerrilla wrote: post #29

Earlier quoted context omitted.

What the fuck. If they're investing that much then why don't they just go straight to formal verification. This is what things like frama-c (or whatever's popular now) are for.

The problem is that effort to do formal verification goes exponential beyond a certain point. seL4 is around 10-12 KLoC, and it took a decade of effort from multiple people to make it happen. At the size of SQLite, especially where they have to operate on platforms with different behavior (as an OS, seL4 is the platform), formal verification is just too much effort. All that said, your reaction is totally understanda…

Link to how SQLite is tested, for anyone who's curious: https://www.sqlite.org/testing.html

There's also an interesting thing where formal verification requires a formal specification, which afaik there isn't one for SQLite. One of the toughest problems that someone would run into trying to put together a formal specification for code as widely deployed as SQLite boils down to Hyrum's Law[1]: on a long enough time scale, all observable behaviours of your system become interfaces that someone, somewhere depends on.

That massive suite of test cases isn't a formal specification but given that it achieves 100% branch coverage that implies to me that it:

- pretty tightly bounds the interface without formally specifying it

- also pretty tightly constrains the implementation to match the current implementation

Which, when you have as many users as you do with SQLite, is probably a fair way of providing guarantees to your users that upgrading from 3.44.0 to 3.45.1 isn't going to break anything you're using unless you were relying on explicitly-identified buggy behaviour (you'd be able to see the delta in the test cases if you looked at the Fossil diffs).

[1] https://www.hyrumslaw.com

Post reply on HN