Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

111–120 of 377 posts

Re: I Can’t Write My Name in Unicode

#111

Post by jes5199 »

microcolonel wrote: post #34

“Whatever path we take, it’s imperative that the writing system of the 21st century be driven by the needs of the people using it. In the end, a non-native speaker – even one who is fluent in the language – cannot truly speak on behalf the monolingual, native speaker.” Not sure how the author can simultaneously say this, while criticizing the CJK unification, which makes total sense, and has never been a point of con…

I'm having trouble finding the reference now, but years ago I did read a story about how an early version of Unicode's reference document depicted CJK characters in a font with Chinese-styled characters, causing slowed adoption in Japan who had a gut reaction that "Unicode is too Chinese". Later editions printed several variations of characters, and the objections mostly evaporated.

(Apologies if I've gotten the countries reversed in this story, it's been years and google is failing me)

Re: I Can’t Write My Name in Unicode

#112

Post by peterfirefly »

mbillie1 wrote: post #58
exelius wrote: post #45

Earlier quoted context omitted.

Right, but some languages are insanely complex to implement. It might be a better idea to teach English to people around the globe rather than cater to every individual need (which will still leave people unable to communicate across languages). I'm not saying other languages should go away -- but the world would also benefit from having a "universal" language, which is more or less English at this point (Mandarin is…

> some languages are insanely complex to implement I don't understand this, is there more to implementing a language than creating glyphs for its character set? I wouldn't think the linguistic complexity would matter at all, only the number of glyphs in the 'alphabet' or similar?

Yes, if the way the characters look depends on the other characters in the word including some that are nowhere near being neighbours. Especially if there were weird and complex rules about how this changed depending on the kind of word being written.

There are a number of writing systems that are evil like that, including several Indic ones.

The code that handles this complicated process is often called a shaper. Choosing and combining the correct glyphs involves a complicated dance between that and the font(s), possibly including large tables (and code!) in the font itself on top of what the shaper does.

Re: I Can’t Write My Name in Unicode

#113

Post by dalke »

mbillie1 wrote: post #58
exelius wrote: post #45

Earlier quoted context omitted.

Right, but some languages are insanely complex to implement. It might be a better idea to teach English to people around the globe rather than cater to every individual need (which will still leave people unable to communicate across languages). I'm not saying other languages should go away -- but the world would also benefit from having a "universal" language, which is more or less English at this point (Mandarin is…

> some languages are insanely complex to implement I don't understand this, is there more to implementing a language than creating glyphs for its character set? I wouldn't think the linguistic complexity would matter at all, only the number of glyphs in the 'alphabet' or similar?

Languages aren't all just simple alphabets like English. Some languages use ligatures to combine characters. In English, things like 'fi' and 'ffl' can be done almost automatically, and is optional, but other languages have stronger and more important rules.

As a simple example, in German the ligature ß is not a simple ligature for 'ss' but a combination of two previous ligatures; long s with round s ("ſs") and long s with (round) z "ſʒ"). Various spelling reforms have simplified the orthography, but "Maßen" and "Massen" are still different words.

Quoting from a Wikipedia page, "Urdu (one of the main languages of South Asia), which uses a calligraphic version of the Arabic-based Nasta`liq script, requires a great number of ligatures in digital typography. InPage, a widely used desktop publishing tool for Urdu, uses Nasta`liq fonts with over 20,000 ligatures"

Then there are rules for presentation. "Complex text layout ... refers to the typesetting of writing systems in which the shape or positioning of a grapheme depends on its relation to other graphemes." - http://en.wikipedia.org/wiki/Complex_text_layout . Cursive English is closest we have to complex text layout; while there are "cursive" fonts where each of the characters is in cursive the letters don't merge. Now imagine a language where smooth connections and fancy curlicues in the "right" places were essential for being seen as erudite, and where "right" depended on 5 years of learning.

Re: I Can’t Write My Name in Unicode

#114

Post by ohjesusthatguy »

jsmthrowaway wrote: post #39
breadbox wrote: post #20

Earlier quoted context omitted.

Imagine if the letter Q had been left out of Unicode's Latin alphabet. The argument against it is that it can be written with a capital O combined with a comma. (That's going to play hell with naive sorting algorithms, of course, but oh well.) Oh, and also imagine your name is Quentin.

My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" Maybe that's just me. I look forward to the future where…

"My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" "

I could tell you, but I'll need $18,000 first.

Re: I Can’t Write My Name in Unicode

#115

Post by mrgriscom »

Someone1234 wrote: post #98
mmastrac wrote: post #65

Earlier quoted context omitted.

Yeah, I thought that "No native English speaker would ever think to try “Greco Unification”" was a poor argument. In seems like a reasonable idea.

I think their argument was: The characters look the same (e.g. Russian's first character and the English A) but have different meanings. So in this example if you searched for the English word "Eat" that is also a completely legal Russian word (E, A, and T, exist in English and Russian), however it means nothing remotely similar. I don't know if they're right or wrong. I am just saying that might be the point they we…

I think the real reason was to preserve round-trip compatibility (legacy char -> unicode char -> legacy char) with the existing encodings for those alphabets.

Re: I Can’t Write My Name in Unicode

#116

Post by evincarofautumn »

microcolonel wrote: post #34

“Whatever path we take, it’s imperative that the writing system of the 21st century be driven by the needs of the people using it. In the end, a non-native speaker – even one who is fluent in the language – cannot truly speak on behalf the monolingual, native speaker.” Not sure how the author can simultaneously say this, while criticizing the CJK unification, which makes total sense, and has never been a point of con…

Not to say there aren’t problems with CJK unification. It’s difficult or impossible to represent old family names or newly coined characters without some means of composing characters from radicals—even if you do want “precomposed” characters a majority of the time, as is the case with, say, é (00E9) versus é (0065 0301).

As far as these characters being “the same”, I think it’s better to say they’re analogues. A in English is analogous to A in Polish—both are Latin alphabets, but neither is the Latin alphabet, and these languages use “the same” letter quite differently. And, say, Å is a letter in Swedish, but in English it’s A with a separate accent mark.

These are also just as well analogous to Α in Greek or А in Cyrillic, so hell, I would actually support the author’s “Greco Unification” strawman if it could be done in a principled way.

Re: I Can’t Write My Name in Unicode

#117

Post by sdg1 »

chimeracoder wrote: post #92
sdg1 wrote: post #48

Not sure if the l33tspeak analogy is fully justified. In case of the "missing" letter (called khanda-ta in Bengali) for the Bengali equivalent of "suddenly", historically, it has been a derivative of the ta-halant form (ত + ্ + ‍ ). As the language evolved, khanda-ta became a grapheme of its own, and Unicode 4.1 did encode it as a distinct grapheme. A nicely written review of the discussions around the addition can b…

> I could write the author's name fine: আদিত্য Author here. Well, yes and no. The jophola at the end is not actually given its own codepoint[0]. The best analogy I can give is to a ligature in English[1]. The Bengali fonts that you have installed happen to render it as a jophola, the way some fonts happen to render "ff" as "ff" but that's not the same thing as saying that it actually is a jophola (according to the Uni…

> The Bengali fonts that you have installed happen to render it as a jophola

It's not only the Bengali font - the text rendering framework of my operating system also needs to have a bunch of complex rules to figure out that a jophola needs to be rendered. It also needs to know that the visual ordering of i-kar is before the preceding consonant cluster (দ in আদিত্য).

> the characters that are required to type a jophola have no semantic, phonetic, or orthographic connection to the jophola.

Not so sure about that. The fact that it's called a "jo"-phola points to a relationship. The relationship may have become less apparent as the script has evolved (though there are words such as সহ্য which makes the relationship more visible), but the distinction is still not as pronounced as between "ta" and "khanda-ta". For the khanda-ta case, it was explicit from the the then-current editions of the dictionaries produced by the language bodies of both Bangladesh and West Bengal that the character had become distinct (স্বতন্ত্র বর্ণ was the phrase that was used). As far as I know, there hasn't been any such claim about jophola from the language bodies. Also, if you look at the collation in Bengali dictionaries, jo-phola is treated as (্+য) for collation.

Re: I Can’t Write My Name in Unicode

#118

Post by VLM »

a2tech wrote: post #90

Ok, we all understand it sucks. Whats the fix? You think Unicode is racist and terrible. Its stupidly difficult to work with for sure, but racist is a stretch. What do you propose? More complexity layered on a system that people don't understand isn't really a fix.

Also they organizational mystery of how to open the floodgates for unpopular languages while still keeping Klingon and Tolkien Elvish out is something of a mystery.

Re: I Can’t Write My Name in Unicode

#119

Post by mikeash »

mlmonkey wrote: post #21
a_c_s wrote: post #10

Earlier quoted context omitted.

This is a terrible excuse: the Unicode Consortium should always seek out at least one (if not a group of) native speakers of a language before defining code points for that language. These speakers really should be both native speakers of and experts in the language. There are countless ways to reach out to Bengali speakers, only one of which is the Indian government - whatever politics governments may play, a techno…

That's ridiculous. How can you expect UC to reach out to the 1000s of languages out there? Plus, shouldn't the party who expects to benefit put in the effort? Did you know that the Indian Government has a department for just such a thing, http://tdil.mit.gov.in/ ? What has it been doing all these years? Added later: there's also CDAC http://cdac.in/index.aspx?id=mlingual . And these are just 2 that I found quickly.

How can you not expect UC to reach out to the 1000s of languages out there, when their entire and sole reason to exist is to allow computers to work with all of those languages?

There's no point in taking on a job and then declaring it to be too difficult.

Re: I Can’t Write My Name in Unicode

#120

Post by baddox »

ptx wrote: post #99
microcolonel wrote: post #34

“Whatever path we take, it’s imperative that the writing system of the 21st century be driven by the needs of the people using it. In the end, a non-native speaker – even one who is fluent in the language – cannot truly speak on behalf the monolingual, native speaker.” Not sure how the author can simultaneously say this, while criticizing the CJK unification, which makes total sense, and has never been a point of con…

I agree, but it seems tricky like it would be tricky to strike exactly the right balance between unifying too much and too little. The arguments for Han unification could have just as well been applied to unifying the Nordic languages – the Swedish Ä is really exactly the same letter as Danish/Norwegian Æ (except that Swedish words never ever use the latter and presumably Danish words never use the former), so it cou…

> I agree, but it seems tricky like it would be tricky to strike exactly the right balance between unifying too much and too little.

That may be true, but not unifying at all must certainly fall under unifying too little.

Post reply on HN