Unicode is harder than you think
mcilloni.ovh
Unicode is harder than you think
1–10 of 121 posts
Re: Unicode is harder than you think
#2It's not that it's hard, it's just people don't go out of their way to escape UTF-8 glyphs into ASCII when dealing with exotic glyphs in a text editor. It's more mundane and tedious, but not 'hard'.
Try working with raw UTF-8 in JS and find yourself in a world of pain. Mathias Bynens talks about these gotchas here:
Re: Unicode is harder than you think
#3Favourite unicode fact: properly rendering unicode requires understanding of the current geopolitical situation (Depending on whom you accept as a country and whom you do not, two country-code-letters may or may not render as a flag. This changes sometimes in today's world.). https://esham.io/2014/06/unicode-flags
Re: Unicode is harder than you think
#4No kidding, you have not lived until you try and explain UTF-8 to people who only believes in what they called "doublebyte".
You think they get it, but surprise happens when a database load fails when loading Chinese Character "string" into a field sized calculated based upon 2 bytes per character.
Re: Unicode is harder than you think
#5If you found this essay interesting, you owe it to yourself to check out this super entertaining talk "Plain Text"[0] from NDC 2022 by Dylan Beattie. Rabbit hole warning: This video caused me to lose an entire Sunday watching Dylan's talks on YouTube, which are uniformly awesome.
Re: Unicode is harder than you think
#6TIL of UTF-1, what an odd specification.
Re: Unicode is harder than you think
#7If you found this essay interesting, you owe it to yourself to check out this super entertaining talk "Plain Text"[0] from NDC 2022 by Dylan Beattie. Rabbit hole warning: This video caused me to lose an entire Sunday watching Dylan's talks on YouTube, which are uniformly awesome. [0]: https://www.youtube.com/watch?v=gd5uJ7Nlvvo
I also really enjoy Dylan Beattie's work. For those with some spare time, who might like to see a true "rockstar" programmer, you may like "The Art of Code"[0].
Re: Unicode is harder than you think
#8No kidding, you have not lived until you try and explain UTF-8 to people who only believes in what they called "doublebyte". You think they get it, but surprise happens when a database load fails when loading Chinese Character "string" into a field sized calculated based upon 2 bytes per character.
Thank god for emojis! Those people would say, "No one in our org would use chinese" and refuse to fix things... but now I just point them to latest message from upper management which contain emoji or two.
(And emoji are such a fine example - once they ate on the table, you need support for combining characters, characters outside of BMP, ligatures.. a large part of Unicode spec)
Re: Unicode is harder than you think
#9Favourite unicode fact: properly rendering unicode requires understanding of the current geopolitical situation (Depending on whom you accept as a country and whom you do not, two country-code-letters may or may not render as a flag. This changes sometimes in today's world.). https://esham.io/2014/06/unicode-flags
Interesting. They pushed all the politics onto the font designers.
Re: Unicode is harder than you think
#10No kidding, you have not lived until you try and explain UTF-8 to people who only believes in what they called "doublebyte". You think they get it, but surprise happens when a database load fails when loading Chinese Character "string" into a field sized calculated based upon 2 bytes per character.
It's terrible, and we IMHO owe that to some introductory university courses to Java (plus some Win32 veterans). I got very close to being rejected by a professor that was obstinately convinced that Unicode "characters" were 2 bytes because it drunk the Kool Aid in the '90s about Java's `char` type representing a Unicode character. Ugh. I still get angry by thinking back at that sometimes