Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

361–370 of 377 posts

Re: I Can’t Write My Name in Unicode

#361

Post by azinman2 »

This is one of the most interesting HN post+comments I've read yet, in part because it mixes technology with culture and history. It also takes advantage of the diverse HN community and their own native languages.

As an American (English speaker) who has studied French, Hebrew and Japanese, I can appreciate the complexity of input balanced against standards and the needs of programmers.

It's a fucking hard problem, but I don't think that blaming the unicode consortium is the right place to do so. They seem to be doing a reasonably good job in trying to get everything in, and really they need input from outsiders to do this well. It requires people with linguistic & technical backgrounds which is probably why random governments may have a harder time providing input.

Further from all the points people are making about uppercase/lowercase/hyphenation across languages, it sounds to me like there really needs to be a super-standarized open source implementation of the things you want to do with text, not just purely encode it. I don't think that exists, and it might be a good place for the unicode people to branch to.

Re: I Can’t Write My Name in Unicode

#362

Post by eloisant »

SiVal wrote: post #311
mbessey wrote: post #261

Earlier quoted context omitted.

I respectfully disagree. If Japanese ideograms and Chinese ideograms actually used different code points (i.e. no "Han unification"), then the problem wouldn't exist - the phone could trivially use a Japanese font for Japanese text, and a Chinese font for Chinese text.

No. Using different code points for the same character used in different languages creates big problems. It would be like having different code points for 'A' depending on whether it was used in English, Spanish, German, etc. If you somehow ended up writing "color" with both 'o' characters from the Spanish ABCs and the others from the English ABCs, you'd have a real mess when it came to sorting, searching, name match…

When you phrase it this way, "using different code for the same characters" it sounds obvious, but the problem precisely is whether they are the same characters or not. Are they like sans serif (used in English) or Gothic (traditionally used in German), or are they like the Roman alphabet and the Cyrillic alphabet, different but from the same root?

Gather 5 linguists, you won't get them to agree on that. Unicode says they're the same, but not everyone agrees with them, and the practical problems are real.

Re: I Can’t Write My Name in Unicode

#363

Post by webitorial »

drawkbox wrote: post #162

It is much easier to criticize than to fix it. While it is good to bring awareness to this, we are still growing in this area. In fact we should applaud the efforts so far that we even have a standard that somewhat works for most of the digital world. Does it need to evolve further, yes. I am sure the engineers and multi-lingual people that stepped up to do Unicode and organize it aren't trying to exclude anyone. Lar…

Generally true, but the problem here is not an issue of bandwidth or racism. Unicode can represent this character, but does so with two codepoints, a technical decision the author doesn't feel is useful. He blames this on the dominance of white people in the work (a questionable assumption, given he didn't link to the extensive list of international liaisons and international standards bodies). The participants in Unicode, including a native Bengali speaker who responded above, considered the argument presented but chose a different path to be consistent with how other characters are treated. The author needs to more carefully distinguish the codepoint, input, and rendering issues raised in his argument.

Re: I Can’t Write My Name in Unicode

#364

Post by webitorial »

PSeitz wrote: post #206

The problem here is not that these people are white, it's the languages they are not speaking.

And a native Bengali speaker has discussed his input into the Unicode discussions and why he disagreed with the author. What do you know about the linguistic input into the work or the participants? Did you check the huge international lists of liaisons that the author ignored? It costs $75 to join Unicode.

Re: I Can’t Write My Name in Unicode

#365

Post by webitorial »

4bpp wrote: post #302

I'd imagine that from the point of view of a Unicode consortium member, the question as to whether to include a particular Bengali glyph some argued to be obsolete looks more like "should we lower the threshold for what characters in a language are considered deserving of a separate codepoint, potentially exposing ourselves to a deluge of O(#codepoints/#characters per language) requests for obscure variant characters…

Had nothing to do with proving they were not humorless fascists. There was a legitimate need for a universal codepoint among Japanese cellphone operators. Keep in mind, Japanese is a language where frequently entire concepts are represented in a single character, so this isn't perhaps as odd as you think. "Poo" had a specific semantic in "cellphone Japanese" that the market demanded. To be used interoperably with other Unicode characters, various 'emoji' were added to Unicode. I, for one, retain the right to be called a humorless fascist.

Re: I Can’t Write My Name in Unicode

#366

Post by webitorial »

wbkang wrote: post #249
zhemao wrote: post #227

Earlier quoted context omitted.

> Talking about Bengali being the seventh largest native language, then saying that US$18k is too expensive for a stake in solving this problem? West Bengal and Bangladesh aren't exactly the richest places in the world. > Emojis with skin tones, something every human can use, that have arrived before a character that only Bengalis use is taken as an "outright insult"? Nobody is being inconvenienced by the inability t…

$18k is really a small amount of money for any governmental organization including those places. Even North Korea participates in the process.

Any Bengali scholar can join the work for $75.

Re: I Can’t Write My Name in Unicode

#367

Post by dang »

ohjesusthatguy wrote: post #352
ohjesusthatguy wrote: post #351

Earlier quoted context omitted.

Just because I'm harsh doesn't mean I don't love you guys :)

Hey how do I get downvotes? No fair! :) Answer: I could tell you, but you need 2070 karma first :) :)

From https://news.ycombinator.com/newsguidelines.html:

"Resist commenting about being downvoted. It never does any good, and it makes boring reading.

Please don't bait other users by inviting them to downvote you."

Re: I Can’t Write My Name in Unicode

#368

Post by webitorial »

zhemao wrote: post #277
zhemao wrote: post #276

Earlier quoted context omitted.

Except that the whole purpose of Unicode is to create a character encoding that "enables people around the world to use computers in any language" - taken from the Unicode Consortium website. Bengali is also not an obscure language. It is the 10th most spoken language in the world and the national language of Bangladesh.

And besides, the argument isn't "Why aren't all Bengali characters represented when all English characters are?" The argument is "Why aren't all Bengali characters represented when a pile of poo is?"

No, the argument is "Why didn't the Unicode authors make the same technical choice I would have based on my limited knowledge of this topic -- including not knowing that the pile of poo was created for use in the Japanese market; as well as not knowing that I could participate myself for just $75, not the $18,000 lie in my story; and not understanding the nuances of international standardization; or reviewing the list of international liaisons to the Unicode organization where much of the language-specific work is done?"

Re: I Can’t Write My Name in Unicode

#369

Post by [deleted] »

[deleted] wrote: post #301
MichaelGG wrote: post #144

Earlier quoted context omitted.

Was British rule actually a net negative, in retrospect? Have there been studies done using objective criteria (not emotional) over counties that were colonies versus ones that weren't? I suppose you can't really quantify the value of people that were destroyed by colonization, but you can look at the current population. Also I just gotta wonder: suppose European or other relatively simple-to-encode languages didn't…

[deleted]

[deleted]

Re: I Can’t Write My Name in Unicode

#370

Post by [deleted] »

mbessey wrote: post #346
SiVal wrote: post #311

Earlier quoted context omitted.

No. Using different code points for the same character used in different languages creates big problems. It would be like having different code points for 'A' depending on whether it was used in English, Spanish, German, etc. If you somehow ended up writing "color" with both 'o' characters from the Spanish ABCs and the others from the English ABCs, you'd have a real mess when it came to sorting, searching, name match…

Actually, Cyrillic is an interesting case. The Unicode standard does define completely-separate codepoints for the Cyrillic letters, even for the ones that look "just like" letters of the Latin Alphabet. Greek letters that look exactly like Latin letters get the same treatment. It's difficult to come up with a logical explanation for why European languages that use their own alphabet get their own codepoints, but ide…

[deleted]
Post reply on HN