Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Escaping user input is ridonkulously hard

codeofhonor.substack.com

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Escaping user input is ridonkulously hard

#51

Post by Arnavion »

tyingq wrote: post #39
tialaramex wrote: post #34

Earlier quoted context omitted.

If the language explicitly says how strings are defined, libraries that go "Eh, I'll just shove nonsense bytes in this data structure and claim that's a string" are broken by definition. That's just as true in Java as in Rust. The problem is languages like C++ or D which just don't care and have a "string" type that might just be some bytes.

I don't mean libraries, I mean external services. Ambiguous strings are everywhere.

The libraries in question are the ones consuming the output of those external services. If an external service sends data that does not map to the programming language's string type, then the string type will fail to be created from that invalid input, and the library was wrong to have tried.

Re: Escaping user input is ridonkulously hard

#52

Post by tyingq »

Arnavion wrote: post #51
tyingq wrote: post #39

Earlier quoted context omitted.

I don't mean libraries, I mean external services. Ambiguous strings are everywhere.

The libraries in question are the ones consuming the output of those external services. If an external service sends data that does not map to the programming language's string type, then the string type will fail to be created from that invalid input, and the library was wrong to have tried.

The external services are not always either explicit, or compliant about the content of said string. Follow this comment chain up, and you'll see mention of blinding casting to utf-8. The point I was making is that you don't always know what the encoding is.

Re: Escaping user input is ridonkulously hard

#53

Post by Arnavion »

tyingq wrote: post #52
Arnavion wrote: post #51

Earlier quoted context omitted.

The libraries in question are the ones consuming the output of those external services. If an external service sends data that does not map to the programming language's string type, then the string type will fail to be created from that invalid input, and the library was wrong to have tried.

The external services are not always either explicit, or compliant about the content of said string. Follow this comment chain up, and you'll see mention of blinding casting to utf-8. The point I was making is that you don't always know what the encoding is.

If the service doesn't send you back a string it doesn't send you back a string. End of story. You're continuously trying to complicate the issue by insisting that they're sending strings and that the client needs to guess their encoding. They are not and it does not.

Re: Escaping user input is ridonkulously hard

#54

Post by tyingq »

Arnavion wrote: post #53
tyingq wrote: post #52

Earlier quoted context omitted.

The external services are not always either explicit, or compliant about the content of said string. Follow this comment chain up, and you'll see mention of blinding casting to utf-8. The point I was making is that you don't always know what the encoding is.

If the service doesn't send you back a string it doesn't send you back a string. End of story. You're continuously trying to complicate the issue by insisting that they're sending strings and that the client needs to guess their encoding. They are not and it does not.

Encoding was one example. But "string" is language dependent aside from encoding. Strong typing doesn't solve the inherent issue.

Re: Escaping user input is ridonkulously hard

#55

Post by 8n4vidtmkvmk »

evilDagmar wrote: post #40
ezfe wrote: post #27

Earlier quoted context omitted.

no, the backend has no reason to see `foo%20bar` - you escape when you're combining that string with other strings (ie into HTML, into a SQL query, etc.)

There's just a whole library of CVEs for people who attempt to escape things being sent to SQL. Use parameterized queries already.

every one says "just use parameterized queries" but they don't handle arrays which makes the idea rather useless.

Re: Escaping user input is ridonkulously hard

#56

Post by 8n4vidtmkvmk »

shadowgovt wrote: post #33

This is a space where type systems can be extremely helpful. Escaped input and unescaped input are separate types. And a robust type system will allow you to craft your functions so that the streams cannot be crossed without going through translation layers. In fact, the most robust type systems will offer things like automatic function composition so that you have to write a minimum of code... If a type coercion fun…

+1 wrapping safe strings with a type is the way to go if you need to compose things together

Re: Escaping user input is ridonkulously hard

#57

Post by 8n4vidtmkvmk »

evilDagmar wrote: post #41

I've always found it more useful to just discard user input that doesn't come in the format you're asking for, and bail on the entire operation. Like, if the user might be attempting something fishy, there's no reason to try and "clean it up" and have your program "do it's best" with the remainder. Throw an error back at the user and move on to the next query.

that sounds awful. you probably reject phone numbers that use spaces instead of dashes or something? if its correctable, just correct it and don't hassle the user. if its ambiguous, then fine, ask the user to clarify.

Re: Escaping user input is ridonkulously hard

#58

Post by DeathArrow »

I use a strongly typed language, repository pattern and an ORM. Good luck trying SQL injections. Also input is sanitized at framework level so good luck with XSS.

Also the input has to bypass validation (for which I have unit tests) and the DTOs are mapped to database models before being written.

Re: Escaping user input is ridonkulously hard

#59

Post by teddyh »

tyingq wrote: post #54
Arnavion wrote: post #53

Earlier quoted context omitted.

If the service doesn't send you back a string it doesn't send you back a string. End of story. You're continuously trying to complicate the issue by insisting that they're sending strings and that the client needs to guess their encoding. They are not and it does not.

Encoding was one example. But "string" is language dependent aside from encoding. Strong typing doesn't solve the inherent issue.

Strong typing never “solved” anything, except that is forces you to see the problem and solve it yourself, explicitly, instead of relying on weak typing to fudge the types for you.

Re: Escaping user input is ridonkulously hard

#60

Post by teddyh »

masklinn wrote: post #47
teddyh wrote: post #44

Earlier quoted context omitted.

> Instead you should escape everything, and opt-out as early as possible. That’s not even remotely workable for any system with more than one kind of “escaping”. What if I want to use a string as: 1. An IDNA-encoded domain name 2. An HTML text snippet 3. A shell command string argument 4. A string literal part of a regular expression 5. A part to be used in an XML CDATA section 6. A JSON string I can’t escape the str…

> I can’t escape the string beforehand, since the escaping rules are all different. You’re still misunderstanding. You shouldn’t escape at any point, instead you should mark things as safe as early as possible. “Safe” almost always has a single context, you don’t care if it’s going to go somewhere else because it’s not safe for there. Anything that’s not marked as safe is then automatically considered unsafe and proc…

> It’s not an issue, because by default nothing is safe anywhere, so all those APIs should treat the injected data thus.

No library does this, since it does not know what strings I send it with their literal meaning intended, and which strings I send it with their escape characters intended to be interpreted. The escape characters are part of the API of that library. The library does not accept “strings” as such, it accepts “escaped” strings. And since my program deals with normal unescaped strings, I have to escape the strings before I send them to the API.

> There is no escaping, because everything is automatically internally escaped by default.

I have a feeling that you have a different meaning of the word “escaped” than me.

Post reply on HN