Live data from Hacker News

If your code accepts URIs as input, filter out “file://”

blog.steve.fi

71–80 of 161 posts

Re: If your code accepts URIs as input, filter out “file://”

#71
post #69
post #39

Earlier quoted context omitted.

Well, I found the same by pure chance before reading this article, and I suspect many more in the HN crowd did. If already half a dozen people on HN report they’ve found it and emailed the person about it, it’s likely it’s too late for responsible disclosure.

Agreed! Better get that karma before someone else does /snark

The issue is more that if so many people have already found it, who else has?

Re: If your code accepts URIs as input, filter out “file://”

#72

Earlier quoted context omitted.

>Wrong way around: only allow http:// and https:// For myself, it's a subtle change in developer thinking - "what should I allow" vs "what should I exclude" - that's paid off massively over the years.

https://en.m.wikipedia.org/wiki/Robustness_principle

It should be pointed out that while this was once accepted as gospel, it has been coming under a lot of fire lately. HTML, once arguably the flagship of this principle and its greatest success (I say "arguably" because you can also argue TCP), no longer works this way. HTML5 specifies how bad input should be handled, and if you accept that "how to process nominally bad input" as the "real" standard, HTML is now strict in what it accepts. It's just that what it is strictly accepting appears quite flexible.

I'm not a big believer in it myself; "liberal in what you accept" and "comprehensible for security audits" are not quite directly opposed, but certainly work against each other fairly hard. There's a time and a place for Postel's principle, but I consider it more an exception for exceptional circumstances rather than the first thing you reach for.

Re: If your code accepts URIs as input, filter out “file://”

#73

Earlier quoted context omitted.

I certainly am weary of bug branding...

I only started seeing this weary/wary misspelling in recent years. They don't sound alike, and they don't really look alike. Did cell phone spellcheckers give rise to this one?

'weary' means tired of something, 'wary' means cautious of something.

Re: If your code accepts URIs as input, filter out “file://”

#74

Earlier quoted context omitted.

I only started seeing this weary/wary misspelling in recent years. They don't sound alike, and they don't really look alike. Did cell phone spellcheckers give rise to this one?

'weary' means tired of something, 'wary' means cautious of something.

Right, and in the last year or so I've started to see people getting these terms confused.

Re: If your code accepts URIs as input, filter out “file://”

#75
post #32
post #9

Earlier quoted context omitted.

Exactly. Whitelist only trusted schemes, do not wait to blacklist untrusted. I wrote the Go HTML sanitizer: https://github.com/microcosm-cc/bluemonday and have a rule for user generated (untrusted) content that basically does whitelist just the things that one can trust: https://github.com/microcosm-cc/bluemonday/blob/master/helpe... That states that URIs must be: 1. Parseable 2. Relative 3. Or one of: mailto http ht…

Might want to add tel to the whitelist. It works in roughly the same way as mailto but interfaces with telephone apps instead of email clients.

This is the default user-generated policy, others are able to tweak and adjust using policy rules, i.e:

    p.AllowURLSchemes("tel")
I chose conservative and safe defaults, not everyone wishes to whitelist telephone links.

Re: If your code accepts URIs as input, filter out “file://”

#76
Don't write server code that opens URI's that come in as input, period. If you take URI's as input, do it only to turn them around and spit them out into some Javascript sent back to the same session. Whatever can or cannot be accessed this way is the browser's problem.

Re: If your code accepts URIs as input, filter out “file://”

#77

Earlier quoted context omitted.

Right, best to check the whole URL. Something like "http/../../../etc/passwd" might get through otherwise.

Like.... $url = rawurldecode($_GET['url']); $url_without_protocol = str_replace(array('https://', 'http://'), '', $url); $protocol = (stristr($url, 'https://') ? 'https' : 'http'); $page = file_get_contents($protocol . '://' . $url_without_protocol);

What about

    u = urlparse(url)
    if u.scheme not in ['http', 'https']:
        return 400

Re: If your code accepts URIs as input, filter out “file://”

#78
post #33

Earlier quoted context omitted.

Can we please stop trying to enumerate badness [1]? When parsing input it is possible to define the set of valid input, not all possible invalid inputs. Also, anybody accepting input from an untrusted source (such as anything from a network or the user) that isn't verifying the data with a formal recognizer is doing it wrong[2]. Instead of writing another weird machine, guarantee that the input is valid with a parser…

guarantee that the input is valid with a parser generator OK, that works really well... until you learn how much non-RFC-specified behavior is built in to web browsers. Simply building a parser to the RFC will leave you wide open to all sorts of nastiness! The is_safe_url() internal function in Django is a bit of a historical dive into things we've learned about how browsers interpret (or, arguably, misinterpret) var…

> non-RFC-specified behavior

I never said anything about limiting the parser to what's defined in an RFC. The acceptable input to "quirks mode" is just another (non-RFC) grammar, which still needs to be defined and validated.

Re: If your code accepts URIs as input, filter out “file://”

#79
post #51

Earlier quoted context omitted.

Right but how does the user submit it and what do you put in the href?

If a user copied the URL from the address bar, it will be correctly percent-encoded already. You can put the same percent-encoded URL in the href attribute of a hyperlink. A properly encoded URL will not contain any character that requires escaping in an HTML context. When a user clicks on that link, the browser will navigate to the percent-encoded URL but display the snowman icon in the address bar. If the user copi…

I just tried doing that with a few domain names containing an umlaut (äöü) and every single time that letter was copied into the clipboard (even though behind the scenes at the request level it would have been encoded). This is what I expect as a regular user. They don't want to deal with encoded, unreadable URLs.
Post reply on HN