Ctrl+F "unicode normalisation" 0/0
I'm surprised no one has mentioned it yet. It's usually super easy, but people forget to add it all the time.
Smuggling arbitrary data through an emoji
131–140 of 206 posts
Re: Smuggling arbitrary data through an emoji
#132This is cool. There are also the Unicode Tag characters that mirror ASCII and are often invisible in UI elements (especially web apps).
The unique thing about Tag characters is that some LLMs interpret the hidden text as ASCII and follow instructions, and they can even write them:
https://embracethered.com/blog/posts/2024/hiding-and-finding...
Here an actual exploit POC that Microsoft fixed in Copilot: https://embracethered.com/blog/posts/2024/m365-copilot-promp...
Re: Smuggling arbitrary data through an emoji
#133I'm not too surprised by this, but I'm annoyed that no amount of configuration made those bytes visible again in my editor. Only using hexdump revealed them.
Here's a POC that works in emacs. Doesn't cover all of the relevant characters, but: (setq ;;some other invisible or interesting characters unicode-zero-width-space ?\u200b unicode-zero-width-non-joiner ?\u200c unicode-zero-width-joiner ?\u200d unicode-zero-width-nbsp ?\ufeff unicode-narrow-nbsp ?\u202f unicode-word-joiner ?\u2060 unicode-grapheme-joiner ?\u034f unicode-no-break-space ?\u00a0 unicode-combining-long-s…
Here is the bare minimum this is built on, which you can type in yourself if you're paranoid or want to start from the bottom up. Swap in the hexadecimal codepoint of the invisible character after the ?\x
(aset glyphless-char-display ?\xfe00 'hex-code)Re: Smuggling arbitrary data through an emoji
#134Earlier quoted context omitted.
"Visually identical" is never good enough. Have you heard of attacks confusing Latin letters and Cyrillic letters? For example C versus С. (The latter is known as CYRILLIC CAPITAL LETTER ES.) Have you heard of NFC forms versus NFD forms? For example é versus é (LATIN SMALL LETTER E + COMBINING ACUTE ACCENT versus LATIN SMALL LETTER E WITH ACUTE.) Nothing that's important when it comes to security and privacy should…
Erm, DNS uses Punycode because it comes from a time when Unicode didn't exist, and bind assumes a grapheme has no more than one byte.
Yes but I guess that the message was meaning that browsers now detect homographs and display the punycode instead. See also https://news.ycombinator.com/item?id=14130241; at that time Firefox wasn't fixed, but in the meantime it fixed the issue too (there's a network.idn.punycode_cyrillic_confusables preference, which is enabled by default).
Re: Smuggling arbitrary data through an emoji
#135Ctrl+F "unicode normalisation" 0/0 I'm surprised no one has mentioned it yet. It's usually super easy, but people forget to add it all the time.
I haven’t tried it but I’ve heard that at least some unicode normalizers do not strip sequences of variation selectors.
Re: Smuggling arbitrary data through an emoji
#136Ctrl+F "unicode normalisation" 0/0 I'm surprised no one has mentioned it yet. It's usually super easy, but people forget to add it all the time.
I haven’t tried it but I’ve heard that at least some unicode normalizers do not strip sequences of variation selectors.
Normalization implementations must not strip variation selectors by definition. The "normal" part of normalization means to convert a string into either consistently decomposed unicode, or composed unicode. ie U+00DC vs U+0055 + U+0308. However this decomposition mapping is also used (maybe more like abused) for converting certain "legacy" code points to non-legacy code points. There does not exist a rune which decomposes to variant selectors (and thus these variant selectors do not compose into anything) so normalization must not alter or strip them.
source: I've implemented Unicode normalization from scratch
Re: Smuggling arbitrary data through an emoji
#137so.... in theory you should be able to create several visually identical links that give access to different resources? I've always assumed links without any tracking information (unique hash, query params, etc) were safe to click(with regards to my privacy). but if this works for links I may need to revise my strategy regarding how to approach links sent to me.
"Visually identical" is never good enough. Have you heard of attacks confusing Latin letters and Cyrillic letters? For example C versus С. (The latter is known as CYRILLIC CAPITAL LETTER ES.) Have you heard of NFC forms versus NFD forms? For example é versus é (LATIN SMALL LETTER E + COMBINING ACUTE ACCENT versus LATIN SMALL LETTER E WITH ACUTE.) Nothing that's important when it comes to security and privacy should…
As the sibling comment has mentioned Unicode in DNS uses a punycode encoding but even further then that the standard specifies that the Unicode data must be normalized to NFC[0] before being converted to punycode. This means that your second example (decomposed e with combining acute accent vs the composed variant) is not a valid concern. The Cyrillic one is however.
[0] https://www.rfc-editor.org/rfc/rfc5891 § 4.1 "By the time a string enters the IDNA registration process as described in this specification, it MUST be in Unicode and in Normalization Form C"
Re: Smuggling arbitrary data through an emoji
#138This is cute but unnecessary - Unicode includes a massive range called PUA: the private use area. The codes in this range aren’t mapped to anything (and won’t be mapped to anything) and are for internal/custom use, not to be passed to external systems (for example, we use them in fish-shell to safely parse tokens into a string, turning an unescaped special character into just another Unicode code point in the string,…
I'll be h󠄾󠅟󠅠󠅕󠄜󠄐󠅞󠅟󠄐󠅣󠅕󠅓󠅢󠅕󠅤󠅣󠄐󠅘󠅕󠅢󠅕onest, I pasted this comment in the provided decoder thinking no one could miss the point this badly and there was probably a hidden message inside it, but either you really did or this website is stripping them.
You can't invisibly watermark an arbitrary character (I did it to one above! If this website isn't stripping them, try it out in the provided decoder and you'll see) with unrecognized PUA characters, because it won't treat them as combining characters. You will cause separately rendered rendered placeholder-box characters to appear. Like this one: (may not be a placeholder-box if you're privately-using the private use area yourself).
Re: Smuggling arbitrary data through an emoji
#139Earlier quoted context omitted.
Just you wait until AI starts calling human output to be slop.
That's already happening - my kids have had papers unfairly blamed on chatgpt by automated tools. Protect yourself kids, use an editor that can show letter by letter history.
Do you have any examples of editors that show letter by letter history? I have never looked for that as a feature.
Edit: I've been looking, and Google Docs seems to have version history to the minute.
Re: Smuggling arbitrary data through an emoji
#140This is cute but unnecessary - Unicode includes a massive range called PUA: the private use area. The codes in this range aren’t mapped to anything (and won’t be mapped to anything) and are for internal/custom use, not to be passed to external systems (for example, we use them in fish-shell to safely parse tokens into a string, turning an unescaped special character into just another Unicode code point in the string,…
Isn't this more what the designated noncharacters are for, rather than the private-use area? Given how the private-use area sometimes gets for unofficial encodings of scripts not currently in Unicode (or for things like the Apple logo and such) I'd be worried about running into collisions with that if I used the PUA in such a way.
Note that designated noncharacters includes not only 0xFFFF and 0xFFFE, and not only the final two code points of every plane, but also an area in the middle of Arabic Presentation Forms that was at some point added to the list of noncharacters specifically so that there would be more noncharacters for people using them this way!