41–50 of 88 posts
Like, if the user might be attempting something fishy, there's no reason to try and "clean it up" and have your program "do it's best" with the remainder. Throw an error back at the user and move on to the next query.
Earlier quoted context omitted.
> That doesn't make sense to me and I agree with GP. If I consume HTML and I escape all HTML input I'm given, I'm utterly useless. [...] Now when I consume text and convert that text into HTML for further treatment, I'm producing HTML, and I must properly escape my input in that conversion. Which is my point, it's the consumption side which defines what the escaping should be. > Escaping only ever occurs at a boundar…
I disagree, a database interface is a format boundary at which a transformation occurs (from text to SQL) and so is globbing (from text to pattern).
Point doesn't change: what escaping is needed is a function of the "transformer" and applied to the input (= consumption) side of it.
You don't apply an escaping because data comes from a database, you apply it because it goes into one. Same with template processors, regex engines, etc...
An alternate view: “string” is not a granular enough type, just like “bitfield” is not a type. Firstly, a string could be raw unknown bytes, verified UTF-8, or UCS-2 (or even UTF-16 or UCS-4), and you absolutely need to know which it is. But let’s assume that you’ve been a diligent programmer and filtered all that at the edges, and now have a sequence of Unicode code points (or possibly graphemes). You still need to…
Escape strings at the last possible moment, and ideally it's done by whatever library you're using so you never have to worry about it. It's never not been clear to me in our codebases if I'm dealing with a raw string or a safe one. They're all unsafe, because you have no clue what context they're going to be used in. If you're writing a web framework or a DB library things might be different though - in that case a…
That's correct, but it's the reverse thinking from the escaping one.
Because in the escaping one, when you need not to escape you will also not-escape at the last possible moment, and that's a sure-fire way to launder attacker-controlled data.
Instead you should escape everything, and opt-out as early as possible.
> But if you're writing a webapp, passing around escaped strings is a bad idea 99% of the time. It creates code highly coupled to one aspect of your system.
That's why you do the reverse: most strings are unsafe to everything, but the strings which are safe are generally safe to one specific subsystem. So you say that.
> Just imagine if you did this with networking. I'm glad we're not in a world where we're passing around TCPString or UDPString or IPString or EthernetString or TokenRingString or CarrierPigeonString because that happens to be a networking stack the app uses sometimes. It sounds like hell.
It sounds like hell because it makes no sense, there's no such thing as a TCPString because TCP is not string-based and TCP messages are not composed that way.
Earlier quoted context omitted.
Escape strings at the last possible moment, and ideally it's done by whatever library you're using so you never have to worry about it. It's never not been clear to me in our codebases if I'm dealing with a raw string or a safe one. They're all unsafe, because you have no clue what context they're going to be used in. If you're writing a web framework or a DB library things might be different though - in that case a…
> They're all unsafe, because you have no clue what context they're going to be used in. That's correct, but it's the reverse thinking from the escaping one. Because in the escaping one, when you need not to escape you will also not-escape at the last possible moment, and that's a sure-fire way to launder attacker-controlled data. Instead you should escape everything, and opt-out as early as possible. > But if you're…
That’s not even remotely workable for any system with more than one kind of “escaping”. What if I want to use a string as:
1. An IDNA-encoded domain name
2. An HTML text snippet
3. A shell command string argument
4. A string literal part of a regular expression
5. A part to be used in an XML CDATA section
6. A JSON string
I can’t escape the string beforehand, since the escaping rules are all different. No, the only sensible alternative is to use the same rule which we all use for character encoding: Encode and decode (and escape) at the edges.
An alternate view: “string” is not a granular enough type, just like “bitfield” is not a type. Firstly, a string could be raw unknown bytes, verified UTF-8, or UCS-2 (or even UTF-16 or UCS-4), and you absolutely need to know which it is. But let’s assume that you’ve been a diligent programmer and filtered all that at the edges, and now have a sequence of Unicode code points (or possibly graphemes). You still need to…
There is no such thing as an "escaped string". Escaping is not a general concept, it is something that differs given the intended destination of that string. For example, "I am a %SYSTEM% person" is a perfectly fine escaped bash string, but an unsafe CMD string; it is also fine as a C# format string, but potentially unsafe as a C format string, depending on your actual implementation of printf; it is an escaped MSSQL…
It's not, though. It's the easiest thing in the world: Just use a library that never emits unescaped content by default, or if you make a single-character typo. The problem is that most of the libraries aren't that.
> It's the easiest thing in the world: Just use a library that never emits unescaped content by default That doesn't make any sense? Escaping is a function of the consumer, not the producer. Hell, most of the problematic content doesn't come from a library to start with. And if your Markdown -> HTML converter produces escaped content... it's not a Markdown -> HTML converter, because the result is not HTML. More broad…
Earlier quoted context omitted.
> They're all unsafe, because you have no clue what context they're going to be used in. That's correct, but it's the reverse thinking from the escaping one. Because in the escaping one, when you need not to escape you will also not-escape at the last possible moment, and that's a sure-fire way to launder attacker-controlled data. Instead you should escape everything, and opt-out as early as possible. > But if you're…
> Instead you should escape everything, and opt-out as early as possible. That’s not even remotely workable for any system with more than one kind of “escaping”. What if I want to use a string as: 1. An IDNA-encoded domain name 2. An HTML text snippet 3. A shell command string argument 4. A string literal part of a regular expression 5. A part to be used in an XML CDATA section 6. A JSON string I can’t escape the str…
You’re still misunderstanding. You shouldn’t escape at any point, instead you should mark things as safe as early as possible.
“Safe” almost always has a single context, you don’t care if it’s going to go somewhere else because it’s not safe for there.
Anything that’s not marked as safe is then automatically considered unsafe and processed as such by the sink.
> What if I want to use a string as:
It’s not an issue, because by default nothing is safe anywhere, so all those APIs should treat the injected data thus.
There is no escaping, because everything is automatically internally escaped by default.
Earlier quoted context omitted.
Escape strings at the last possible moment, and ideally it's done by whatever library you're using so you never have to worry about it. It's never not been clear to me in our codebases if I'm dealing with a raw string or a safe one. They're all unsafe, because you have no clue what context they're going to be used in. If you're writing a web framework or a DB library things might be different though - in that case a…
> They're all unsafe, because you have no clue what context they're going to be used in. That's correct, but it's the reverse thinking from the escaping one. Because in the escaping one, when you need not to escape you will also not-escape at the last possible moment, and that's a sure-fire way to launder attacker-controlled data. Instead you should escape everything, and opt-out as early as possible. > But if you're…
I've always found it more useful to just discard user input that doesn't come in the format you're asking for, and bail on the entire operation. Like, if the user might be attempting something fishy, there's no reason to try and "clean it up" and have your program "do it's best" with the remainder. Throw an error back at the user and move on to the next query.
Earlier quoted context omitted.
Too bad if your consumer has to interact with anything that could be malicious in any circumstances! Consumers must properly escape any input.
That doesn't make sense to me and I agree with GP. If I consume HTML and I escape all HTML input I'm given, I'm utterly useless. Now when I consume text and convert that text into HTML for further treatment, I'm producing HTML, and I must properly escape my input in that conversion. The escaping is only needed because I produce HTML. In fact the only time escaping can be done is when producing data, because if unesca…
The thing that accepts the input must make sure it is properly escaped. Think of SQL injection attacks - they are because the thing that accepts input hasn't properly escaped the input.
Cross site scripting attacks are exactly the same thing but occur when the input side doesn't properly escape HTML input.