I Can’t Write My Name in Unicode • Hacker News Board

Re: I Can’t Write My Name in Unicode

#41

Post by hawkice » Tue, Mar 17, 2015, 6:24 PM UTC

Getting rid of CJK unification would better model actual language change in the future (France, for instance, has a group that keeps a rigorous definition of the French language up to date -- I would enjoy giving them a subset of the codes to define how to write French).

But the general principle sounds odd. Should 家, the simplified Chinese character and 家, the traditional Chinese character have different codepoints? Should no French be written using characters with lower, English code points because of their need for a couple standard characters? Should latin be written using a whole new set of code points even though it needs no code points not contained in ascii?

Re: I Can’t Write My Name in Unicode

#42

Post by fixermark » Tue, Mar 17, 2015, 6:25 PM UTC

gerbal wrote: post #16

nemo wrote: post #4

I wonder if the author has submitted a proposal to get the missing glyph for their name added. You don't need to be a member of the consortium to propose adding a missing glyph/updating the standard. The point of the committee as I understand it isn't to be an expert in all forms of writing, but to take the recommendations from scholars/experts and get a working implementation, though more diverse representation of l…

Holy cow CJK unification is a terrible idea. Maybe if it originated from the CJK governments, it might be an OK idea, but the idea of a Western multinationals trying to save Unicode space by disregarding the distinctness of a whole language group is idiotic. The fundamental roll of an institution like the Unicode Consortium is to be descriptive, not prescriptive. If there is a human script, passing certain, low, low…

I don't disagree, while observing that it plays hell with security if (forgive my use of Latin; I don't have a codepoint translator handy) 'bat.com' and 'bat.com' are two different websites because the 'a' in the first is a Chinese-a and the 'a' in the second is a Korean-a.

(Of course, this calls into question the wisdom of expanding DNS into the Unicode space in the first place---a space that does nothing like guarantee 1-to-1 association between visual glyph and code for an application that has been built on the assumption that different codes are visually distinguishable. But that ship has sailed).

Re: I Can’t Write My Name in Unicode

#43

Post by estebank » Tue, Mar 17, 2015, 6:25 PM UTC

breadbox wrote: post #20

vilhelm_s wrote: post #7

Earlier quoted context omitted.

It sounds like the glyph is in unicode already, but expressed using combining characters?

Imagine if the letter Q had been left out of Unicode's Latin alphabet. The argument against it is that it can be written with a capital O combined with a comma. (That's going to play hell with naive sorting algorithms, of course, but oh well.) Oh, and also imagine your name is Quentin.

> Imagine if the letter Q had been left out of Unicode's Latin alphabet.

To properly write my european last name I have to press between 2 and 4 different simultaneous keys, depending on the system. Han unification is beyond misguided, but combining characters is not the problem.

Re: I Can’t Write My Name in Unicode

#44

Post by bbreier » Tue, Mar 17, 2015, 6:26 PM UTC

nothrabannosir wrote: post #35

bbreier wrote: post #13

Earlier quoted context omitted.

The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i…

tangent, but isn't 国 simplified and 國 traditional guó?

Yes, that's exactly right. In this case, Japanese uses the simplified version, but in others it uses the traditional version (or even a version that is slightly different from the current traditional version used in Taiwan or Cantonese.)

Re: I Can’t Write My Name in Unicode

#45

Post by exelius » Tue, Mar 17, 2015, 6:26 PM UTC

beat wrote: post #11

angersock wrote: post #2

I came in expecting to read an article bemoaning some niche language and playing the diversity card. I was not disappointed, but as I kept reading, the author made some very good points. I don't really care that the organization is run by white men who speak English, because frankly the entire computing industry and telecommunications industry is based on that. I'm not going to argue about the original sin there, bec…

Bengali is the seventh most widely spoken language in the world, with more native speakers than Russian. That's hardly a "niche language". And, as pg has pointed out, making the computer industry English-centric and America-centric is greatly limiting us. Intelligence and will to power are equally distributed across the globe. But institutional barriers to success are not. This is a classic example of an institutiona…

Right, but some languages are insanely complex to implement. It might be a better idea to teach English to people around the globe rather than cater to every individual need (which will still leave people unable to communicate across languages).

I'm not saying other languages should go away -- but the world would also benefit from having a "universal" language, which is more or less English at this point (Mandarin is spoken by more people but is rarely spoken outside of Asia). If we want to maximize intelligence and will to power, it would be best if like-minded individuals could communicate regardless of where they're from or what their native language is.

Re: I Can’t Write My Name in Unicode

#46

Post by microcolonel » Tue, Mar 17, 2015, 6:27 PM UTC

yummyfajitas wrote: post #25

bbreier wrote: post #13

Earlier quoted context omitted.

The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i…

Nor is it unreasonable to "unify" Latin, Greek and Cyrilic: Cyrillic ПФ vs Greek ΠΦ Cyrillic АВ vs Latin AB Obviously using ω for w (as he does) is stupid, but his reducto-ad-absurdum is not particularly absurd.

I think their distinct calligraphic representations mean that these would be destructive; whereas with regard to CJK, the characters are clearly represented the same way between the considered languages.

Re: I Can’t Write My Name in Unicode

#47

Post by fixermark » Tue, Mar 17, 2015, 6:27 PM UTC

yarper wrote: post #37

I Can Text You A Pile of Poo, But I Can’t Write My Name ... - by Aditya Mukerjee on March 17th, 2015 What is the glyph missing from this? I know its not ideal but some uncommon glyphs have always been omitted from charsets, for example ASCII never included http://en.wikipedia.org/wiki/%C3%86 , and it was replaced by "ae" in common language. http://en.wikipedia.org/wiki/Hanlon%27s_razor

This is also why we've (almost) all taken to writing résumé as resume.

We've even managed to build text-based search engines that do a pretty decent job of guessing which one we mean.

Re: I Can’t Write My Name in Unicode

#48

Post by sdg1 » Tue, Mar 17, 2015, 6:27 PM UTC

Not sure if the l33tspeak analogy is fully justified.

In case of the "missing" letter (called khanda-ta in Bengali) for the Bengali equivalent of "suddenly", historically, it has been a derivative of the ta-halant form (ত + ্ + ‍ ). As the language evolved, khanda-ta became a grapheme of its own, and Unicode 4.1 did encode it as a distinct grapheme. A nicely written review of the discussions around the addition can be found here: http://www.unicode.org/L2/L2004/04252-khanda-ta-review.pdf

I could write the author's name fine: আদিত্য. A search with the string in the Bengali version of Wikipedia pulls up quite a few results as well, so other people are writing it too. The final "letter" in that string is a compound character, and there's no clear evidence that it needs to be treated as an independent one. Even while in primary school, we were taught the final "letter" in the author's name as a conjunct. In contrast, for the khanda-ta case, it could be shown that modern Bengali dictionaries explicitly referred to khanda-ta as an independent character.

For me, many of these problems are more of an input issue, than an encoding issue. Non latin languages have had to shoe-horn their script onto keyboard layouts designed for latin-scripts, and that has been always suboptimal. With touch devices we have newer ways to think about this problem, and people are starting to try out things.

[Disclosure: I was involved in the Unicode discussions about khanda-ta (I was not affiliated with a consortium member) and I have been involved with Indic localization projects for the past 15 years]

Re: I Can’t Write My Name in Unicode

#49

Post by gopalv » Tue, Mar 17, 2015, 6:28 PM UTC

theon144 wrote: post #15

I don't understand; I don't feel like character combination using the zero width joiner is on the same level as 13375p34k. It looks like the character just doesn't have a separate code-point, but is instead a composite, but still technically "reachable" from within Unicode, no?

That's effectively the only way to write several indian language with unicode - ZWJ + NZWJ and a decent font which supports all the ligatures.

The recent Bullshit Sans font is a clear example to describe how ligatures works. And for Malayalam there are consonant patterns which do not match itself to the Sanskrit model which makes it rather odd to write half-consonants which are full syllables (വ്യഞ്ജനം +് zwj).

And my name is written with one of those (ഗോപാൽ) & I can't say I'm mad about it because I found Unicode to be elegant in another way.

Somewhere in the early 2000s, I was amazed to find out that the Unicode layouts for Malayalam as bytes in UTF-8 were sortable as-is.

As a programmer, I found that detail of encoding to sort order to be very fascinating as it meant that I had to do nothing to handle Malayalam text in my programs - the collation order is implicit, provided everyone reads the ZWJ and NZWJ in their sorting orders.

Re: I Can’t Write My Name in Unicode

#50

Post by saalweachter » Tue, Mar 17, 2015, 6:29 PM UTC

yummyfajitas wrote: post #25

bbreier wrote: post #13

Earlier quoted context omitted.

The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i…

Nor is it unreasonable to "unify" Latin, Greek and Cyrilic: Cyrillic ПФ vs Greek ΠΦ Cyrillic АВ vs Latin AB Obviously using ω for w (as he does) is stupid, but his reducto-ad-absurdum is not particularly absurd.

(For those who are rusty on their Greek, ω is a lower-case omega, and unrelated to the English/German letter w.)