Live data from Hacker News

Charset="WTF-8"

wtf-8.xn--stpie-k0a81a.com

211–220 of 463 posts

Re: Charset="WTF-8"

#212
post #164

Earlier quoted context omitted.

There's no argument here. We could say it's only for script and alphabets, ok. It includes many undeciphered writing systems from antiquity with only a small handful of extent samples. Should we keep that, very likely to never be used character set, but exclude the extremely popular emojis? Exclude both? Why? Aren't computers capable enough? I used to be on the anti emoji bandwagon but really, it's all indefensible.…

Like how phonetic alphabets save space compared to ideograms by just “write the word how it sounds”, the little SVG-icode would just “write the letter how it’s drawn”

Right. Semantic iconography need not be universal or even formal to be real.

Think of all the symbols notetakers invent; ideographs without even phonology assigned to it.

Being as dynamic as flexible as human expression is hard.

Emojis have even taken on this property naturally. The high-5 is also the praying hands for instance. Culturally specific semantics are assigned to the variety of shapes, such as the eggplant and peach.

Insisting that this shouldn't happen is a losing battle against how humans construct written language. Good luck with that.

Re: Charset="WTF-8"

#213

Earlier quoted context omitted.

IMO, the sin of Unicode is that they didn't just pick local language authorities and gave them standardized concepts like lines and characters, and start-of-language and end-of-language markers. Lots of Unicode issues come from handling languages that the code is not expecting, and codes currently has no means to select or report quirk supports. I suppose they didn't like getting national borders involved in technica…

> start-of-language and end-of-language markers Unicode used to have language tagging but they've been (mostly) deprecated: https://en.wikipedia.org/wiki/Tags_(Unicode_block) https://www.unicode.org/reports/tr7/tr7-1.html

The lack of such markers prevents Unicode from encoding strings of mixed Japanese and Chinese text correctly. Or in the case of a piece of software that must accept both Chinese and Japanese names for different people, Unicode is insufficient for encoding the written forms of the names.

Re: Charset="WTF-8"

#214

Earlier quoted context omitted.

Watch as someone names themselves the bell character, “^G” (ASCII code 7) [1] When they meet people, they tell them their name is unpronounceable, it’s the sound of a PC speaker from the late 20th century, but you can call them by their preferred nickname “beep”. In paper and online forms they are probably forced to go by the name “BEL”. [1] https://en.wikipedia.org/wiki/Bell_character

Or Derek https://www.youtube.com/watch?v=hNoS2BU6bbQ

The interaction brings to mind Grzegorz Brzęczyszczykiewicz:

https://www.youtube.com/watch?v=AfKZclMWS1U

(from the Polish comedy film "How I Unleashed World War II")

Re: Charset="WTF-8"

#215

As someone who really think name field should just be one field with any printable unicode characters, I do wonder what the hell would I need to do if I take customer names in this form, and then my system has to interact with some other service that requires first/last name split, and/or [a-zA-Z] validation, like a bank or postal service. Automatic transliteration seems to be very dangerous (wrong name on bank accou…

You can just show the user the transliteration & have them confirm it makes sense. Always store the original version since you can't reverse the process. But you can compare the transliterated version to make sure it matches. Debit cards a pretty common example of this. I believe you can only have ASCII in the cardholder name field.

I have an ID that transliterated my name, and included the original, but the original contained an obvious typo. I immediately notified the government official, but they refused to fix it. They assured me that only the transliterated name would be used.

Human systems aren't always interested in avoiding or fixing defects.

Re: Charset="WTF-8"

#216
post #127

I'll say it again: this is the consequence of Unicode trying to be a mix of html and docx, instead of a charset. It went too far for an average Joe DevGuy to understand how to deal with it, so he just selects a subset he can handle and bans everything else. HN does that too - special symbols simply get removed. Unicode screwed itself up completely. We wanted a common charset for things like latin, extlatin, cjk, cyri…

But hey, multiocular o!

https://en.wikipedia.org/wiki/Cyrillic_O_variants#Multiocula...

(TL;DR a bored scribe's doodle has a code point)

Re: Charset="WTF-8"

#217

WTF-8 is actually a real encoding, used for encoding invalid UTF-16 unpaired surrogates for UTF-8 systems: https://simonsapin.github.io/wtf-8/

I believe this is what Rust OsStrings are under the hood on Windows.

Which I assume stands for "Windows-Transformation-Format-8(bits)".

Re: Charset="WTF-8"

#218
post #210

Earlier quoted context omitted.

Sure, but removing those wouldn't make Unicode any simpler, they're just character sets. The GP is complaining about things like combining characters and diacritic modifiers, which make Unicode "ugly" but are necessary if you want to represent real languages used by billions of people.

I mean, we can just make the languages simpler? We can also remove all the hundred different ways to pronounce English sounds. All elementary students will thank you for it xD

You can make a language simpler but old books still exist. I guess if we burn all old books and disallow a means to print these old books again, people would be happy?

Re: Charset="WTF-8"

#219
post #3

Earlier quoted context omitted.

Is name validation even possible?

If you just use the {Alphabetic} Unicode character class (100K code points), together with a space, hyphen, and maybe comma, that might get you close. It includes diacritics. I'm curious if anyone can think of any other non-alphabetic characters used in legal names around the world, in other scripts? I wondered about numbers, but the most famous example of that has been overturned: "Originally named X Æ A-12, the chi…

> I'm curious if anyone can think of any other non-alphabetic characters used in legal names around the world, in other scripts?

The Catalan name Gal·la is growing in popularity, with currently 1515 women in the census having it as a first name in Spain with an average age of 10.4 years old: https://ine.es/widgets/nombApell/nombApell.shtml

Re: Charset="WTF-8"

#220
post #204
post #148

Earlier quoted context omitted.

The “invisible symbols” are necessary to correctly represent human language. For instance, one of the most infamous Unicode control characters — the right-to-left override — is required to correctly encode mixed Latin and Hebrew text [1], which are both scripts that you mentioned. Besides, ASCII has control characters as well. The “colorful icons” are not part of Unicode. Emoji are just characters like any other. The…

Copy this󠀠󠀼󠀼󠀼󠀠󠁉󠁳󠀠󠁴󠁨󠁩󠁳󠀠󠁮󠁥󠁣󠁥󠁳󠁳󠁡󠁲󠁹󠀠󠁴󠁯󠀠󠁣󠁯󠁲󠁲󠁥󠁣󠁴󠁬󠁹󠀠󠁲󠁥󠁰󠁲󠁥󠁳󠁥󠁮󠁴󠀠󠁨󠁵󠁭󠁡󠁮󠀠󠁬󠁡󠁮󠁧󠁵󠁡󠁧󠁥󠀿󠀠󠀾󠀾󠀾 sentence into this site and click Decode. (YMMW) https://embracethered.com/blog/ascii-smuggler.html

Wow. Did not expect you can just hide arbitrary data inside totally normal looking strings like that. If I select up to "Copy thi" and decode, there's no hidden string, but just holding shift+right arrow to select just "one more character", the "s" in "this", the hidden string comes along.
Post reply on HN