I've built a tool specifically to test if these kind of characters will reach API backends: https://github.com/Endava/cats. My idea was that APIs should explicitly reject or sanitise input containing such characters.
Invisible Characters
31–36 of 36 posts
Re: Invisible Characters
#32This is another good reason to have a text editor you really trust, which can show you these things. Whether it's different line-endings or weird invisible space stuff, I know I can just open it in Vim and figure out what's really going on pretty quickly. Wasted a lot of time earlier in my life on that nonsense (:
Vim does not display them all. The only program I checked which displays all such characters is `less -U`. You can test using this file:
https://gitlab.com/nervuri/nervuri.net/-/raw/master/gopher/z...
Re: Invisible Characters
#33Zero-width characters can be used to covertly watermark text and to figure out who copied text from a page and pasted it somewhere else. Server software can encode a hidden number between every few words, which corresponds to a server log entry with your username (if logged in), IP address, browser fingerprint, etc. I wrote more about this here:
I think the best solution to this type of problem would be a clipboard utility that warns you when you copy text which contains hidden characters, homoglyps, rarely used whitespace characters, etc.
Re: Invisible Characters
#34This is another good reason to have a text editor you really trust, which can show you these things. Whether it's different line-endings or weird invisible space stuff, I know I can just open it in Vim and figure out what's really going on pretty quickly. Wasted a lot of time earlier in my life on that nonsense (:
Vim does not display them all. The only program I checked which displays all such characters is `less -U`. You can test using this file: https://gitlab.com/nervuri/nervuri.net/-/raw/master/gopher/z...
Thanks for this! Good to add to the ol' repertoire (:
Looks like the only one Vim misses is U+17B5? Though it there could be more not listed there. Unicode is a deep dark forest.
----
For others readers, here's a non-gopher version of the article linked inside: https://nervuri.net/stega
Re: Invisible Characters
#35My favorite is U+202E Right-to-Left Override, which doesn't appear to be listed there. A surprising amount of UIs (apps, sites) can be broken with it as they were never tested with right-to-left writing direction in mind. Even a Unicode reference website that I just used to recall the code is broken by it. [0] Entering RLO into arbitrary input forms for fun can bend spacetime, I swear. [0] https://unicode-table.com/e…
> Even a Unicode reference website that I just used to recall the code is broken by it.
I like to imagine that sites like this have noticed the bug but have the sense of humor to choose not to fix it.
Re: Invisible Characters
#36Earlier quoted context omitted.
Vim does not display them all. The only program I checked which displays all such characters is `less -U`. You can test using this file: https://gitlab.com/nervuri/nervuri.net/-/raw/master/gopher/z...
Thanks for this! Good to add to the ol' repertoire (: Looks like the only one Vim misses is U+17B5? Though it there could be more not listed there. Unicode is a deep dark forest. ---- For others readers, here's a non-gopher version of the article linked inside: https://nervuri.net/stega
Also 061C, E0001, E0020...E007F. And probably others, yes. The list at https://invisible-characters.com/ might contain more.
> Unicode is a deep dark forest.
Oh, indeed.