Earlier quoted context omitted.
Actually yes, the CJK unification is a problem for many people, including me when I want to read Japanese on a phone bought in Europe. Example 1. Typically, any time you want to mix the 2 languages you're getting in trouble. Let's say you write a textbook for Chinese people to learn Japanese as a second language. Or a research article in Japanese citing old Chinese literature. In your text, you'll have to mark specif…
Neither of your examples are solveable by the Unicode consortium, who are not the god-emperors of fonts (nor would you want them to be).
I Can’t Write My Name in Unicode
261–270 of 377 posts
Re: I Can’t Write My Name in Unicode
#262Earlier quoted context omitted.
> My point in the original comment (and to some extent in the preceding one) was to emphasize that a lot of these issues are at the input method level - we should not have to think about encoding as long as it accurately and unambiguously represent whatever we want it to represent. I might be sympathetic to this, except that keyboard layouts and input (esp. on mobile devices) is an even bigger mess and even more frag…
Not necessarily disagreeing with your broader point but I just want to point out that the examples are only obscure and antiquared in English. æ is common in modern Danish and unambiguously a separate letter, as is þ in Icelandic.
For Japanese (and Chinese and a few others) things are even more complicated. It's tricky to fit ~5000 symbols on a keyboard, so typically in Japan one types on either a phonetic layout, or a latin layout, and translate to Kanji as needed (eg: "nihongo" or "にほんご" is transformed to "日本語" -- note also that "ご" itself is a compound character, "ko" modified by two dots to become "go" -- which may or may not be entered as a compound, with a modifier key).
As I currently don't have any Japanese input installed under xorg, that last bit I had to cut and paste.
It is entirely valid to view ø as a combination of o and a (short) slash, or å as a combination of "a" and "°" -- but if one does that while typing, it is important that software handles the compound correctly (and distinct from ligatures, as mentioned above). My brother's name, "Ståle" is five letters/symbols long, if reversed it becomes "elåtS", not "el°atS" (six symbols).
So, yeah, it's complicated. Remember that we've spent many years fighting the hack that was ascii, extended ascii (which may be (part of) why eg: Norwegian gets to have å rather than a+°). You still can't easily use utf8 with neither C nor, as I understand it C++ (almost, but not quite -- AFAIK one easy workaround is to use QT's strings if one can have qt as a dependency -- and it's still a mess on Windows, due to their botched wide char hacks... etc).
All in all, while it's nice to think that one can take some modernized, English-centric ideas evolved from the Gutenberg press, and mash it together with what constitutes a "letter" (How hard is it to reverse a string!? How hard is it to count letters!? How hard is it to count words?!) -- that approach is simply wrong.
There will always be magic, and there'll be very few things that can be said with confidence to be valid across all locales. What is to_upper("日本語"), reverse("Ståle"), character_count("Ståle"), word_count("日本語") etc.
This turned into a bit more of an essay than I intended, sorry about that :-)
Re: I Can’t Write My Name in Unicode
#263Earlier quoted context omitted.
My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" Maybe that's just me. I look forward to the future where…
> …including a weekly "worst of HN" comment dissection. That sounds interesting, but I can't find any reference to it on their site or search engines. Do you have a link?
https://web.archive.org/web/20140724125906/http://modelviewc...
When you see a piece on MVC by "The Editors," that's code for Shanley Kane writing under an anonymous byline.
Re: I Can’t Write My Name in Unicode
#264I guess that the main point of the article is that the Unicode Standard is being decided by a bunch of North American and European people, instead of being decided by the speakers of the languages Unicode is intended to.
In my opinion, for instance, Han Unification is a botch and no Japanese would consider that it makes sense.
Re: I Can’t Write My Name in Unicode
#265Getting rid of CJK unification would better model actual language change in the future (France, for instance, has a group that keeps a rigorous definition of the French language up to date -- I would enjoy giving them a subset of the codes to define how to write French). But the general principle sounds odd. Should 家, the simplified Chinese character and 家, the traditional Chinese character have different codepoints?…
There was an academic proposal in the '90's for something called "multicode" (IIRC) that did exactly this: every character had a language associated with it, so there were as many encodings for "a" as there were languages that used "a", and all of them were different, or at least every character was somehow tagged so the language it "came from" was somehow identifiable. Fortunately, it never caught on. The notion tha…
glib white supremacists are the best kind of white supremacists. "it's progress!"
Re: I Can’t Write My Name in Unicode
#266Re: I Can’t Write My Name in Unicode
#267I think the author misses the point completely. Things like this: "No native English speaker would ever think to try “Greco Unification” and consolidate the English, Russian, German, Swedish, Greek, and other European languages’ alphabets into a single alphabet." The author probably ignores that different European languages used different alphabet scripts until very recently. For example, Gothic and other different s…
CJK unification doesn't make learning Chinese characters any easier, it only means that characters that look exactly the same will use the same code point. This is not related to simplifying characters. Also, I regularly see people hand-writing Chinese characters on tablets, so it does happen. And for those who use Pinyin to enter characters, it doesn't matter if the characters are simplified or traditional or unified or whatever, because they only have to pick the right one in their IME.
Re: I Can’t Write My Name in Unicode
#268Its broken technically and a setback socially.
Unicode itself is such a unfathomably huge project that it's impossible to do it right, too many languages, too many weird writing systems, and too many ways to do mathematical notation on paper that can't be expressed. Just look at the code pages, they are an utter mess.
Computers and ASCI were a chance to start anew, to establish english as a universal language, spoken by everybody.
The pressure on governments who would wanted to partake in the digital revolution would have forced them to introduce it as an official secondary language.
Granted english is not the nicest language, but is the best candidate we have in terms of adoption, and relative simplicity (Mandarin is another contester, but several thousand logograms are really impractical to encode.).
Take a look at the open source world, where everybody speaks english and collaborates regardless of nationality. One of the main factors why this is possible, is that we found a common language, forced on us by the tools and programming languages we use.
If humanity wants get rid of wars, poverty and nationalism, we have to find a common language first.
A simple encoding and universal communication is a feature, fragmented communication is the bug.
Besides. UTF-8 is broken because it doesn't allow for constant time random character access and length counting.
Re: I Can’t Write My Name in Unicode
#269Earlier quoted context omitted.
Isn't this the question of a font? In which case the client chooses if they want to use a font with a double-story or a single-story g?
When reading 'gas', the client will have to figure out whether that is about a liquid, in which case it has to choose a colonial 'g' (when written with a British 'g' 'gas' always is a liquid). If the meaning is that of a gaseous substance, the client will have to do additional work to determine what kind of 'g' to write.
Re: I Can’t Write My Name in Unicode
#270Earlier quoted context omitted.
Right, but some languages are insanely complex to implement. It might be a better idea to teach English to people around the globe rather than cater to every individual need (which will still leave people unable to communicate across languages). I'm not saying other languages should go away -- but the world would also benefit from having a "universal" language, which is more or less English at this point (Mandarin is…
It's one thing to say English will be the international language of trade and commerce. It's another thing altogether to say we won't bother making character sets that represent other languages, because it's too hard. Since when has "too hard" ever stopped an engineer? That should be catnip for us!
I'd much rather have something that works and is simple. Than something that makes it easy for everybody and is broken.