Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hidden Messages in Emojis and Hacking the US Treasury

slamdunksoftware.substack.com

Post reply on HN
Print view View original

61–70 of 82 posts

Re: Hidden Messages in Emojis and Hacking the US Treasury

#61

Post by Validark »

whatnow37373 wrote: post #60
Validark wrote: post #54

Could someone please explain to me why "sanitizing database inputs" was ever considered a good idea? Why not just add a feature in SQL like so? SELECT * FROM users WHERE username = [ ]raw-text-of-len-not-parsed-at-all E.g. SELECT * FROM users WHERE username = [21]flyin' and wavin' guy ^^^^^^^^^^^^^^^^^^^^^ these 21 chars are NOT parsed AT ALL, just taken as data I am not very familiar with SQL so you might need a dif…

How is this not "sanitizing inputs"? The basic idea behind your proposal exists and is called prepared statements. It's actually, I hope, the normal way to write queries these days. You write your query like: "SELECT * FROM users WHERE username = ?" and execute your query like "execute(query, username)". The problem? It's optional.

To me, "sanitizing inputs" implies a transformation of the data into a string that can be "safely" evaluated as code which hopefully yields the input data. Instead you should be able to just mark a piece of the code as data, that will never be tokenized or parsed or anything, just dropped directly into a buffer.

"Prepared statements" sounds EXACTLY like what I was thinking! I don't understand why people would ever use anything else.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#62

Post by whatnow37373 »

Validark wrote: post #61
whatnow37373 wrote: post #60

Earlier quoted context omitted.

How is this not "sanitizing inputs"? The basic idea behind your proposal exists and is called prepared statements. It's actually, I hope, the normal way to write queries these days. You write your query like: "SELECT * FROM users WHERE username = ?" and execute your query like "execute(query, username)". The problem? It's optional.

To me, "sanitizing inputs" implies a transformation of the data into a string that can be "safely" evaluated as code which hopefully yields the input data. Instead you should be able to just mark a piece of the code as data, that will never be tokenized or parsed or anything, just dropped directly into a buffer. "Prepared statements" sounds EXACTLY like what I was thinking! I don't understand why people would ever us…

Ah, I see! It's a cool idea, but .. let's try to be maximally obtuse and pedantic today. I'm a developer and it's HN after all.

[4]tree is also code that yields data. At the end of the day some kind of parser needs to decide what to do with your data and [ ] is just another way of escaping special characters. In this case it escapes entire strings instead of individual characters. It's your special way of sanitizing the input.

Questions: Who is responsible for the number? What is this number: bytes, "characters", runes? What happens if the number is wrong? (If you expose this number to external factors of any kind you get a special, interesting new breed of SQL injection.)

In practice you'd probably do something like:

my_special_superduper_safety_syntax_preprocessor("SELECT * FROM users WHERE username=$$$", "peter")

Which will yield something like:

"SELECT * FROM users WHERE username=[5]peter"

.. so you don't have to calculate the number. If we're doing this, why not just go for:

exec("SELECT * FROM users WHERE username=?", "peter")

.. and be done with it.

> I don't understand why people would ever use anything else.

Yes, I agree. Usually it's some interesting combination of laziness and ignorance.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#63

Post by oefrha »

TheDong wrote: post #57
thaumasiotes wrote: post #50

Earlier quoted context omitted.

> The correct way to do something like this will always be parameterized input which looks something like this > Why? Because [] the postgres protocol splits out the command and the data for the command in a way that can't be injected. I'm not sure I'm comfortable with this. You can create a prepared statement and then pass user input to it as parameters, sure. https://www.postgresql.org/docs/17/sql-prepare.html But…

The postgres escape function actually worked fine before this "CVE". It was documented as escaping something for use as part of a postgres query. BeyondTrust used it as input to the 'psql' tool, which is an interactive tool you're not really supposed to programmatically invoke, and the documentation for the postgres escape function didn't say it escaped input for psql. Even though postgres was fine calling it a CVE a…

> If BeyondTrust had just used it as part of a postgres query string, the escape function would have been sufficient.

That's completely false. The following (pseudo code because working with C strings is verbose and beside the point) with nothing to do with psql should be fine:

  PQexec(conn, "SELECT * FROM user WHERE nickname = '" + PQescapeString(user_input) + "';")
but thanks to the vulnerable PQescapeString(), the following user_input

  "\xc0'; DROP TABLE user"
would fuck it up. That's just the failed escape function leading to a classic SQL injection. Using psql makes it worse because psql can execute additional non-SQL commands, but this escape function is not "fine" at all with or without psql.

> With parameterized queries, the escaping and the query parsing are done in the same place

Again, wrong. For parametrized queries, params don't go through serialization because they don't need to hit the parser, there's no "escaping" whatsoever.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#64

Post by account42 »

defen wrote: post #24
netsharc wrote: post #15

Geez, any summary of this article that tells it like the reader isn't five years old?

UTF-8 encodes a unicode codepoint into 1, 2, 3, or 4 bytes. Assuming that you have a valid UTF-8 encoding of a codepoint, then the first byte tells you how many bytes are in the encoding. 0-127 inclusive means one byte, 192-223 means 2, 224-239 means 3, and 240-247 means 4. If the first byte is 0xC0 (192), then the sequence is two bytes long. However, not every 2-byte sequence that starts with 0xC0 is valid UTF-8. Th…

The funny part is that not having any Unicode support in this part of the code and treating the data as ASCII (plus mistery bytes) would have worked correctly.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#65

Post by voxic11 »

cogman10 wrote: post #28
0xbadcafebee wrote: post #21

Earlier quoted context omitted.

The only reason BeyondTrust implemented that was it wasn't untrusted user commands . They sanitized the data, so it should have been fine. The unfortunate problem was that the sanitizer didn't sanitize. Systems are built on a set of expectations. Undermine the expectations and you undermine the system.

> They sanitized the data, so it should have been fine. This is a 101 rookie level approach to SQL or injection defense. It's dumb for exactly the same reason why this is dumb "SELECT * FROM foo WHERE bar=" + sanitize(userInput) The correct way to do something like this will always be parameterized input which looks something like this "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); Why? Because that the…

This is the article I link my developers whenever I see them making this mistake "Don’t try to sanitize input. Escape output." https://benhoyt.com/writings/dont-sanitize-do-escape/

Its been fairly effective at making them realize the fundamental mistake they are making. Quoting the key part:

> The only code that knows what characters are dangerous is the code that’s outputting in a given context.

> So the better approach is to store whatever name the user enters verbatim, and then have the template system HTML-escape when outputting HTML, or properly escape JSON when outputting JSON and JavaScript.

> And of course use your SQL engine’s parameterized query features so it properly escapes variables when building SQL

Re: Hidden Messages in Emojis and Hacking the US Treasury

#66

Post by TheDong »

oefrha wrote: post #63
TheDong wrote: post #57

Earlier quoted context omitted.

The postgres escape function actually worked fine before this "CVE". It was documented as escaping something for use as part of a postgres query. BeyondTrust used it as input to the 'psql' tool, which is an interactive tool you're not really supposed to programmatically invoke, and the documentation for the postgres escape function didn't say it escaped input for psql. Even though postgres was fine calling it a CVE a…

> If BeyondTrust had just used it as part of a postgres query string, the escape function would have been sufficient. That's completely false. The following (pseudo code because working with C strings is verbose and beside the point) with nothing to do with psql should be fine: PQexec(conn, "SELECT * FROM user WHERE nickname = '" + PQescapeString(user_input) + "';") but thanks to the vulnerable PQescapeString(), the…

> but thanks to the vulnerable PQescapeString(), the following user_input would fuck it up

Nope. To quote: https://www.rapid7.com/blog/post/2025/02/13/cve-2025-1094-po...

> Because of how PostgreSQL string escaping routines handle invalid UTF-8 characters, in combination with how invalid byte sequences within the invalid UTF-8 characters are processed by psql

If you just pass it to postgres over a normal query, postgres will reject an invalid byte sequence in the query with an error, refuse to even parse the query, and thus you won't get a SQL injection. It's just that psql didn't hard-error on invalid utf-8, even though postgres did.

That's why the escape function was suitable for postgres, both the escape function and postgres's query parser assume invalid byte sequences are, you know, invalid.

> For parametrized queries, params don't go through serialization because they don't need to hit the parser, there's no "escaping" whatsoever.

You're right of course, I used imprecise language that everyone understands, and you're choosing to read critically in order to be combative.

You don't have to be so combative.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#67

Post by oefrha »

TheDong wrote: post #66
oefrha wrote: post #63

Earlier quoted context omitted.

> If BeyondTrust had just used it as part of a postgres query string, the escape function would have been sufficient. That's completely false. The following (pseudo code because working with C strings is verbose and beside the point) with nothing to do with psql should be fine: PQexec(conn, "SELECT * FROM user WHERE nickname = '" + PQescapeString(user_input) + "';") but thanks to the vulnerable PQescapeString(), the…

> but thanks to the vulnerable PQescapeString(), the following user_input would fuck it up Nope. To quote: https://www.rapid7.com/blog/post/2025/02/13/cve-2025-1094-po... > Because of how PostgreSQL string escaping routines handle invalid UTF-8 characters, in combination with how invalid byte sequences within the invalid UTF-8 characters are processed by psql If you just pass it to postgres over a normal query, postg…

Sure, the blog post didn't mention PQexec would reject it, so I assumed it would be accepted. Turns out it's a narrowly dodged bullet, I'm wrong on that. But having to chain two vulnerabilities together to own the system doesn't make either vulnerability less of a vulnerability. The escape function was wrong, period, another level of defense helped in this case, but "the documentation for the postgres escape function didn't say it escaped input for psql" is a bullshit excuse (it definitely didn't achieve the documented goal of "escaping special characters so that they cannot cause any harm"), putting "CVE" in quotes and blaming it all on the user is wrong.

> I used imprecise language that everyone understands, and you're choosing to read critically in order to be combative.

No, your "imprecise language" is a fundamental and quite dangerous misunderstanding that could easily lead to more vulnerabilities like this one ("PQexecParams = PQexec + PQescapeString, amiright? I'll just use the latter"). Maybe you didn't misunderstand yourself, maybe you did, but it's 100% misleading for readers not familiar with db internals.

Re: Hidden Messages in Emojis and Hacking the US Treasury

#68

Post by gcr »

cogman10 wrote: post #28
0xbadcafebee wrote: post #21

Earlier quoted context omitted.

The only reason BeyondTrust implemented that was it wasn't untrusted user commands . They sanitized the data, so it should have been fine. The unfortunate problem was that the sanitizer didn't sanitize. Systems are built on a set of expectations. Undermine the expectations and you undermine the system.

> They sanitized the data, so it should have been fine. This is a 101 rookie level approach to SQL or injection defense. It's dumb for exactly the same reason why this is dumb "SELECT * FROM foo WHERE bar=" + sanitize(userInput) The correct way to do something like this will always be parameterized input which looks something like this "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); Why? Because that the…

I’m starting to think that string-based languages like SQL which mix structure and content are a mistake.

Maybe future database systems will only accept queries serialized from protobuf, or JSON (output by a proper serializer)

Re: Hidden Messages in Emojis and Hacking the US Treasury

#69

Post by nthingtohide »

Terr_ wrote: post #13

> In order to to this, PQescapeStringInternal must call pg_utf_mblen. For a moment I had a dev-flashback to the problem of utf8mb4 versus (broken) utf8 in mySQL. Easy now, this is Postgres , everything is safe(er)...

Oh, the fun we had when a single emoji which crashed youtube or whatsapp.

https://www.youtube.com/watch?v=jC4NNUYIIdM

Re: Hidden Messages in Emojis and Hacking the US Treasury

#70

Post by TeMPOraL »

voxic11 wrote: post #65
cogman10 wrote: post #28

Earlier quoted context omitted.

> They sanitized the data, so it should have been fine. This is a 101 rookie level approach to SQL or injection defense. It's dumb for exactly the same reason why this is dumb "SELECT * FROM foo WHERE bar=" + sanitize(userInput) The correct way to do something like this will always be parameterized input which looks something like this "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); Why? Because that the…

This is the article I link my developers whenever I see them making this mistake "Don’t try to sanitize input. Escape output." https://benhoyt.com/writings/dont-sanitize-do-escape/ Its been fairly effective at making them realize the fundamental mistake they are making. Quoting the key part: > The only code that knows what characters are dangerous is the code that’s outputting in a given context. > So the better appr…

> and then have the template system HTML-escape when outputting HTML, or properly escape JSON when outputting JSON and JavaScript

Or, stop using stringly template systems, and treat the data as what it is: a structured language, with well-defined grammar.

One of these days I need to write an article titled "Don't play with escaping strings. Serialize output.". Core idea being, "escaping your output" still looks too much like "sanitizing input"[0], and one tiny mistake is all it takes to give an attacker ability to inject arbitrary code into the page (or give an unlucky user ability to brick the page for themselves) - so instead of working in "string space", work in whatever semantics your output is, and treat the string form as a serialization problem. In case of HTML, that means constructing tree of tags as data structure, and then serializing them. Then, bugs in serializer notwithstanding, the whole class of injection problem disappears - you can't do "$text" -> "", when your "template" is made of data structures like [:h1, $text], because $text can't possibly alter the structure here. Etc.

In some sense, "Don't escape, serialize instead" is the complement of "Parse, don't validate".

(See also: make invalid states unrepresentable.)

--

[0] - Who ever sanitizes input? I've only ever seen this kind of sanitization the article describes happen in the output, within string-gluing templates.

Post reply on HN