Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Escaping user input is ridonkulously hard

codeofhonor.substack.com

Post reply on HN
Print view View original

31–40 of 88 posts

Re: Escaping user input is ridonkulously hard

#31

Post by tyingq »

jerf wrote: post #24
FreezerburnV wrote: post #22

Earlier quoted context omitted.

A type system isn't going to save you from users submitting all kinds of potentially different encodings. Which also depends on what kind of user input is being handled: Is it OS-provided UI? Is it something being sent to a service accessible on the internet? Is it from a CLI? Is it from a file? Context matters for the potential space of what kind of data you might be operating on, which could require different ways…

"A type system isn't going to save you from users submitting all kinds of potentially different encodings." Yes, it is, because you give that a type that indicates you don't know what the encoding is, like RawInput or something. You then can not pass this type to any other function that doesn't explicitly call for that type. If you have some function that accepts it, blindly casts it to UTF-8, and slams it out into a…

>If you have some function that accepts it, blindly casts it to UTF-8

Unfortunately, if you interact with services you didn't write, you're usually back to getting "strings" of unknown encoding, and typically requirements that force some blind or semi-blind guessing.

Re: Escaping user input is ridonkulously hard

#32

Post by bcrosby95 »

teddyh wrote: post #7

An alternate view: “string” is not a granular enough type, just like “bitfield” is not a type. Firstly, a string could be raw unknown bytes, verified UTF-8, or UCS-2 (or even UTF-16 or UCS-4), and you absolutely need to know which it is. But let’s assume that you’ve been a diligent programmer and filtered all that at the edges, and now have a sequence of Unicode code points (or possibly graphemes). You still need to…

Escape strings at the last possible moment, and ideally it's done by whatever library you're using so you never have to worry about it. It's never not been clear to me in our codebases if I'm dealing with a raw string or a safe one. They're all unsafe, because you have no clue what context they're going to be used in.

If you're writing a web framework or a DB library things might be different though - in that case a different class probably makes sense. If you have a module for a certain communication medium, then yeah you might use it in that module. But if you're writing a webapp, passing around escaped strings is a bad idea 99% of the time. It creates code highly coupled to one aspect of your system.

Just imagine if you did this with networking. I'm glad we're not in a world where we're passing around TCPString or UDPString or IPString or EthernetString or TokenRingString or CarrierPigeonString because that happens to be a networking stack the app uses sometimes. It sounds like hell.

Re: Escaping user input is ridonkulously hard

#33

Post by shadowgovt »

This is a space where type systems can be extremely helpful.

Escaped input and unescaped input are separate types. And a robust type system will allow you to craft your functions so that the streams cannot be crossed without going through translation layers.

In fact, the most robust type systems will offer things like automatic function composition so that you have to write a minimum of code... If a type coercion function is available, the type system can be taught to just automatically apply that coercion function before dropping the string into the relevant processing.

Re: Escaping user input is ridonkulously hard

#34

Post by tialaramex »

tyingq wrote: post #31
jerf wrote: post #24

Earlier quoted context omitted.

"A type system isn't going to save you from users submitting all kinds of potentially different encodings." Yes, it is, because you give that a type that indicates you don't know what the encoding is, like RawInput or something. You then can not pass this type to any other function that doesn't explicitly call for that type. If you have some function that accepts it, blindly casts it to UTF-8, and slams it out into a…

>If you have some function that accepts it, blindly casts it to UTF-8 Unfortunately, if you interact with services you didn't write, you're usually back to getting "strings" of unknown encoding, and typically requirements that force some blind or semi-blind guessing.

If the language explicitly says how strings are defined, libraries that go "Eh, I'll just shove nonsense bytes in this data structure and claim that's a string" are broken by definition.

That's just as true in Java as in Rust. The problem is languages like C++ or D which just don't care and have a "string" type that might just be some bytes.

Re: Escaping user input is ridonkulously hard

#35

Post by js2 »

The interesting part of the article is below the fold and not reflected in the headline:

https://codeofhonor.substack.com/i/78789944/security-theater...

"Most of the rendering bugs I’ve seen in security audits don’t matter. This is not how your organization will be pwned. ... What would fix this? Layered security built around a plausible threat model. What would not help? Removing reflected ASCII text from Shodan’s API error message. I’m not saying that small security bugs aren’t worth fixing, or that organizational security always trumps application security. Rather, real damage usually does not come from where security engineers tend to expect, because they spend their time on pentests and CTFs that differ substantially from the approaches popular among actual attackers."

Everyone commenting that dealing with user input is easy: if it were really easy, we wouldn't keep making the same mistakes. I fixed my first SQL injection attack by switching some code to bind variables over 20 years ago, yet we still have Little Bobby Tables showing up in our collective databases. The fix may be easy ("just do X"), but the mistake is even easier.

Re: Escaping user input is ridonkulously hard

#37

Post by AtlasBarfed »

js2 wrote: post #35

The interesting part of the article is below the fold and not reflected in the headline: https://codeofhonor.substack.com/i/78789944/security-theater... "Most of the rendering bugs I’ve seen in security audits don’t matter. This is not how your organization will be pwned. ... What would fix this? Layered security built around a plausible threat model. What would not help? Removing reflected ASCII text from Shodan’s A…

Breadth-first security attacks will exploit input sanitizing exploits like that. Security audits can certainly help with that, assuming they don't impose a huge security infrastructure and review process that crushes developer productivity, which always seems to happen.

Depth-first attacks as described are a different class of attack, and of course "audit" won't help that much. Education, penetration testing, and honeypots are some of the stuff that works for that.

Ultimately, if an organization treats its work force like crap, then depth-first attacks are unstoppable. The crypto-locker attackers are strangely pro-worker, because it highlights how disgruntled employees are such effective attack vectors via bribery, vengeance, or apathy.

Re: Escaping user input is ridonkulously hard

#38

Post by deredede »

masklinn wrote: post #16
deredede wrote: post #13

Earlier quoted context omitted.

That doesn't make sense to me and I agree with GP. If I consume HTML and I escape all HTML input I'm given, I'm utterly useless. Now when I consume text and convert that text into HTML for further treatment, I'm producing HTML, and I must properly escape my input in that conversion. The escaping is only needed because I produce HTML. In fact the only time escaping can be done is when producing data, because if unesca…

> That doesn't make sense to me and I agree with GP. If I consume HTML and I escape all HTML input I'm given, I'm utterly useless. [...] Now when I consume text and convert that text into HTML for further treatment, I'm producing HTML, and I must properly escape my input in that conversion. Which is my point, it's the consumption side which defines what the escaping should be. > Escaping only ever occurs at a boundar…

I disagree, a database interface is a format boundary at which a transformation occurs (from text to SQL) and so is globbing (from text to pattern).

Re: Escaping user input is ridonkulously hard

#39

Post by tyingq »

tialaramex wrote: post #34
tyingq wrote: post #31

Earlier quoted context omitted.

>If you have some function that accepts it, blindly casts it to UTF-8 Unfortunately, if you interact with services you didn't write, you're usually back to getting "strings" of unknown encoding, and typically requirements that force some blind or semi-blind guessing.

If the language explicitly says how strings are defined, libraries that go "Eh, I'll just shove nonsense bytes in this data structure and claim that's a string" are broken by definition. That's just as true in Java as in Rust. The problem is languages like C++ or D which just don't care and have a "string" type that might just be some bytes.

I don't mean libraries, I mean external services. Ambiguous strings are everywhere.

Re: Escaping user input is ridonkulously hard

#40

Post by evilDagmar »

ezfe wrote: post #27
mrkeen wrote: post #3

Earlier quoted context omitted.

Escaping isn't always a yes/no question. If someone enters "foo bar" into your frontend, should the backend only see "foo%20bar" ?

no, the backend has no reason to see `foo%20bar` - you escape when you're combining that string with other strings (ie into HTML, into a SQL query, etc.)

There's just a whole library of CVEs for people who attempt to escape things being sent to SQL. Use parameterized queries already.
Post reply on HN