Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

341–350 of 377 posts

Re: I Can’t Write My Name in Unicode

#341

Post by frivoal »

kalleboo wrote: post #281
frivoal wrote: post #176

Earlier quoted context omitted.

no, it isn't a more apt comparison. Antiqua and Fraktur look much more different from each other than most Chinese and Japanese characters are from eachother. Here are two screenshots in Japanese and Chinese, taken from the http://www.yomiuri.co.jp/ for one, and http://www.cntv.cn/ for the other. Neither media outlet can be accused of being cultural sellouts misrepresenting the written tradition of their countries. J…

Your screenshots are showing the exact same glyphs because you don't have the "宋体" font installed that's specified by the Chinese web page (it's a script font so you'd notice instantly it's different), so your browser picked a Japanese font to render it instead. The only reason they look the same in your screenshots is due to Han Unification. Your reasoning is "they're unified in Unicode which is proof they should be…

The fallback font used on my computer isn't the most appropriate one, but Japanese also uses Song/Ming fonts, and this is much more similar to the difference between a serif and a sans serif font than between different characters.

Re: I Can’t Write My Name in Unicode

#342

Post by fenomas »

anon4 wrote: post #306
mbessey wrote: post #261

Earlier quoted context omitted.

I respectfully disagree. If Japanese ideograms and Chinese ideograms actually used different code points (i.e. no "Han unification"), then the problem wouldn't exist - the phone could trivially use a Japanese font for Japanese text, and a Chinese font for Chinese text.

But.. you already posted a solution to your problem - use chinese font for chinese and japanese font for japanese. There is no problem. I mean, you would also presumably use a western font for latin and another separate font for cyrillic, since japanese fonts universally have absolutely dreadful kerning on latin (and often omit cyrillic entirely).

Having to know metadata about text is precisely what made the pre-unicode days so bad. If you're writing a word document, sure, choose your fonts, but if you're rendering a web page that doesn't happen to declare its language, things aren't so simple. (And if you're writing software meant to correctly handle user input in multiple languages, good luck...)

Re: I Can’t Write My Name in Unicode

#343

Post by elFarto »

jdmichal wrote: post #337
elFarto wrote: post #308

Earlier quoted context omitted.

Actually, that's not correct, and it's the exact same mistake I made when using that API. codePointAt returns the codepoint at index i, where i is measured in 16-bit chars, which means you could index into the middle of a surrogate pair. The correct version is: for (int i = 0; i Java 8 seems to have acquired a codePoints() method on the CharSequence interface which seems to do the same thing. But this just adds to th…

I think you missed the part where `i` is not incremented in the for statement, but inside the loop using `Character.charCount`, which returns the number of `char` necessary to represent the code point. If there's something wrong with this, my unit tests have never brought it up, and I am always sure to test with multi-`char` codepoints.

Your right, I did miss it, I apologize. That'll teach me to read code so early in the morning.

Re: I Can’t Write My Name in Unicode

#344

Post by ticking »

krdln wrote: post #332
ticking wrote: post #321

Earlier quoted context omitted.

English is the best candidate because it has the second largest user base (1.2 Billion vs 1.3 Billion for Mandarin), http://en.wikipedia.org/wiki/List_of_languages_by_total_numb... and is twice as spoken as the third most popular language Spanish. (0.55 Billion) If I got to pick the universal language, it would be Lojban (a few hundred speakers), but that is not a realistic goal, teaching the other 6 Billion people a…

Even if 1.2 billion seems a lot, that's still a small fraction of a world's population. So every choice of a universal language would force majority of a world to learn new one. So that's why I think winning popularity contest is a poor argument and we shouldn't look at that and focus on things like simplicity (which I don't find in English), speed of learning, consistency, expressiveness etc. I'd be happy to use Loj…

With Unicode these predefined functions have to be large and complex. You won't be able to use them on embedded systems for example.

Re: I Can’t Write My Name in Unicode

#345

Post by fenomas »

frivoal wrote: post #336
systoll wrote: post #283

Earlier quoted context omitted.

It applies to Ä and Æ... which is what the parent said. It doesn't apply to 気 and 气 and 氣. Those are all the same thing.

気 气 and 氣 are actually not merged by han unification, and could be described as similar to similar to Ä and Æ: shared etymology, same meaning, some languages decided to use a simpler character because the old one was too complicated to write. The variations on characters which have been merged are usually even closer than that. More like the single or double storey "a", or the single or double loop "g".

Well, maybe if the double-story "a" were used exclusively in one country but considered incorrect in another.

I think what you're missing is that Han unification never particularly concerned itself with whether characters are "the same" or not. Somebody basically just decided that some differences were important and others weren't. I mean, the difference between 語 and 语 is analogous to printing and cursive, but for whatever reason it made the cut. Meanwhile the SC and TC versions of 骨 are approximately mirror images, but to show you I'd need two different fonts.

Anyway the merged differences are not just analogous to different fonts.

Re: I Can’t Write My Name in Unicode

#346

Post by mbessey »

SiVal wrote: post #311
mbessey wrote: post #261

Earlier quoted context omitted.

I respectfully disagree. If Japanese ideograms and Chinese ideograms actually used different code points (i.e. no "Han unification"), then the problem wouldn't exist - the phone could trivially use a Japanese font for Japanese text, and a Chinese font for Chinese text.

No. Using different code points for the same character used in different languages creates big problems. It would be like having different code points for 'A' depending on whether it was used in English, Spanish, German, etc. If you somehow ended up writing "color" with both 'o' characters from the Spanish ABCs and the others from the English ABCs, you'd have a real mess when it came to sorting, searching, name match…

Actually, Cyrillic is an interesting case. The Unicode standard does define completely-separate codepoints for the Cyrillic letters, even for the ones that look "just like" letters of the Latin Alphabet. Greek letters that look exactly like Latin letters get the same treatment.

It's difficult to come up with a logical explanation for why European languages that use their own alphabet get their own codepoints, but ideographic languages need to be "unified", even though the actual letters as used in those languages look different.

The "Han unification" was fundamentally a bad idea, and persists for historical reasons. Back when (some) people thought a fixed-width 16-bit character representation would be "enough", it made sense to try to reduce the number of "redundant" code points. Now that Unicode has expanded to a much-larger code space, I would think they'd choose differently.

Unfortunately, that kind of sweeping change is unlikely any time soon.

Re: I Can’t Write My Name in Unicode

#348

Post by TheCoelacanth »

zhemao wrote: post #276
ramviswanadha wrote: post #142

The argument that author makes is "every letter in English alphabet is represented, why not every letter/grapheme in Bengali/Tamil/Telugu/Name your language" argument is specious at best

Except that the whole purpose of Unicode is to create a character encoding that "enables people around the world to use computers in any language" - taken from the Unicode Consortium website. Bengali is also not an obscure language. It is the 10th most spoken language in the world and the national language of Bangladesh.

Bangladesh is a member of the Unicode Consortium. Why haven't they ensured that Bengali is well supported? The Unicode Consortium's job isn't to support every language in the world; it's to coordinate between the multitude of different parties who do the actual work to ensure that they produce can coexist nicely.

Re: I Can’t Write My Name in Unicode

#349

Post by dspeyer »

mlmonkey wrote: post #21
a_c_s wrote: post #10

Earlier quoted context omitted.

This is a terrible excuse: the Unicode Consortium should always seek out at least one (if not a group of) native speakers of a language before defining code points for that language. These speakers really should be both native speakers of and experts in the language. There are countless ways to reach out to Bengali speakers, only one of which is the Indian government - whatever politics governments may play, a techno…

That's ridiculous. How can you expect UC to reach out to the 1000s of languages out there? Plus, shouldn't the party who expects to benefit put in the effort? Did you know that the Indian Government has a department for just such a thing, http://tdil.mit.gov.in/ ? What has it been doing all these years? Added later: there's also CDAC http://cdac.in/index.aspx?id=mlingual . And these are just 2 that I found quickly.

How about putting in that effort for the 10 languages with over 100 million native speakers?

Re: I Can’t Write My Name in Unicode

#350

Post by ohjesusthatguy »

ohjesusthatguy wrote: post #114
jsmthrowaway wrote: post #39

Earlier quoted context omitted.

My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" Maybe that's just me. I look forward to the future where…

"My first choice as theoretical Quentin wouldn't be "how can I frame this accidental, perhaps even flagrantly disrespectful omission as antiprogressive and dissect the credentials, experience, and ethnicity of the people who made the mistake via culture essay," it would probably be "where do I issue a pull request to fix this mistake or in what way can I help?" " I could tell you, but I'll need $18,000 first.

I absolutely agree with the down-vote; why didn't you white people pull together and work something out for the white guy that has a reasonable pull request for an encoding of a language in which he probably doesn't ever write? Don't y'all un-niggers pull together?
Post reply on HN