Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

251–260 of 377 posts

Re: I Can’t Write My Name in Unicode

#251

Post by mikekchar »

cplease wrote: post #66
icebraining wrote: post #36

Earlier quoted context omitted.

But the letter wasn't left out of Unicode; it's actually typed in the article . It's just internally represented as multiple codepoints, much like one of parts of my name (é) may be. Frankly, this is irrelevant to the actual problem, which is the input system, and which has nothing to do with Unicode. Nothing prevents a single key from typing multiple codepoints at once.

> It's just internally represented as multiple codepoints And in fact it is not, and even in the article it is U+09CE. One codepoint. If his input method irks him, he's as free to tweak it as I am to switch to Dvorak. Also folks, there's no "CJK unification" project. It's Han unification. Han characters are Han characters, just like Latin characters are Latin characters. Just because German has ß and Danish has Ø doe…

I hate to say it, but I think the author's objections seem to stem from his lack of understanding of character encoding issues. I don't know Bengali at all and so I will try to refrain from commenting on it, but I do speak and read Japanese fluently and Han Unification is a very, very good thing. Can you imagine the absolute hell you would have to go through trying to determine if place names were the same if they used different code points for identical characters -- just because of geopolitical origins?

Yes, there are some frustrating issues -- it has been historically difficult to set priorities for fonts in X and since Chinese fonts tend to have more glyphs, you often wind up with Chinese glyphs when you wanted Japanese glyphs. But this is not an encoding issue. Localization/Internationalization is really difficult. Making a separate code point for every glyph is not going to change that for the better.

Re: I Can’t Write My Name in Unicode

#252

Post by krakensden »

eloisant wrote: post #191
microcolonel wrote: post #34

“Whatever path we take, it’s imperative that the writing system of the 21st century be driven by the needs of the people using it. In the end, a non-native speaker – even one who is fluent in the language – cannot truly speak on behalf the monolingual, native speaker.” Not sure how the author can simultaneously say this, while criticizing the CJK unification, which makes total sense, and has never been a point of con…

Actually yes, the CJK unification is a problem for many people, including me when I want to read Japanese on a phone bought in Europe. Example 1. Typically, any time you want to mix the 2 languages you're getting in trouble. Let's say you write a textbook for Chinese people to learn Japanese as a second language. Or a research article in Japanese citing old Chinese literature. In your text, you'll have to mark specif…

Neither of your examples are solveable by the Unicode consortium, who are not the god-emperors of fonts (nor would you want them to be).

Re: I Can’t Write My Name in Unicode

#253

Post by iopq »

yummyfajitas wrote: post #25
bbreier wrote: post #13

Earlier quoted context omitted.

The author's explanation of what characters Chinese, Japanese, and Korean share is very limited. All three languages use Chinese characters in written language to varying extents, and in some cases the differences begin significantly less than a century ago. Though there are cases where the same Chinese character represented in Japanese writing is different from how it is represented in Traditional Chinese writing (i…

Nor is it unreasonable to "unify" Latin, Greek and Cyrilic: Cyrillic ПФ vs Greek ΠΦ Cyrillic АВ vs Latin AB Obviously using ω for w (as he does) is stupid, but his reducto-ad-absurdum is not particularly absurd.

Cyrillic ПФ vs Greek ΠΦ?

Here's Cyrillic lower case: пф Here's Greek lower case: πφ

in some fonts the pi would be rendered with a longer bar on top, but you just showed why it's a bad idea:

I would want to be able to discuss Greek in Russian on a forum, but this would not be possible because all the glyphs in lowercase would look Russian

Re: I Can’t Write My Name in Unicode

#254

Post by hetman »

yuriks wrote: post #106
chimeracoder wrote: post #92

Earlier quoted context omitted.

> I could write the author's name fine: আদিত্য Author here. Well, yes and no. The jophola at the end is not actually given its own codepoint[0]. The best analogy I can give is to a ligature in English[1]. The Bengali fonts that you have installed happen to render it as a jophola, the way some fonts happen to render "ff" as "ff" but that's not the same thing as saying that it actually is a jophola (according to the Uni…

Unicode makes extensive use of combining characters for european languages, for example to produce diacritics: ìǒ or even for flag emoji. A correct rendering system will properly combine those, and if it doesn't then that's a flaw in the implementation, not the standard. It seems like you're trying to single out combining pairs as "less legitimate" when they're extensively used in the standard.

Except unicode also has invididual codepoints for "ǒ" and "ì".

Re: I Can’t Write My Name in Unicode

#255

Post by tracker1 »

hackuser wrote: post #213
tracker1 wrote: post #189

Earlier quoted context omitted.

In your example... I wouldn't really care how it is stored, as long as it looks right on the display, and I don't have to go through contortions to enter it on an input device... for example, I don't care that 'a' maps to \x61 ... it's a value behind the scenes... it's the interface to that value. As long as the typeface/font used can display the character/combination reasonably, and I can input reasonably it doesn't…

> I wouldn't really care how it is stored, as long as it looks right on the display It becomes a problem when you have other uses besides reading text, such as sorting or searching.

That's why you can normalize the input to certain UTF-8 patterns for this purpose. Same should go for passwords before hashing them.

Re: I Can’t Write My Name in Unicode

#256

Post by iso8859-1 »

wbkang wrote: post #248

I see many comments about Han unification being a bad idea but I am not seeing any reason why it was such a bad idea. I am from a CJK country and I find it makes a lot of sense. Most commonly used characters should be considered identical regardless of whether it is used in Chinese Japanese Korean or Vietnamese. Sure there are some characters that are rendered improperly depending on your font but I don't think that…

Is it really depending on the font, or is it depending on some language metadata? Having it depend on the font seems stupid, since a font ideally be able to represent a languages which use a script encodable using Unicode.

Re: I Can’t Write My Name in Unicode

#257

Post by jimktrains2 »

maxlybbert wrote: post #244
nova wrote: post #188

Earlier quoted context omitted.

> ch, ll, ñ, and rr are all considered separate letters In Spanish "rr" has never been considered as a single letter. "Ch" and "ll" used to be, but not anymore. Ñ is, of course.

That's funny; I spent several hours of class time trilling r's to make sure we pronounced "carro" correctly, and repeating a 30 character alphabet.

rr not being its own letter has no bearing on if you can pronounce carro correctly, just like saying church right has no bearing on if c and h are two letters or ch is a single letter.

Re: I Can’t Write My Name in Unicode

#258

Post by Qwertious »

eridius wrote: post #210
exelius wrote: post #110

Earlier quoted context omitted.

They're not unrelated though. You have to have a way to get from your input format to the finished product in a consistent way, and the glyph set you design has a large bearing on that. You can't solve it completely with AI, because then you just have an AI interpretation of human language, not human language. A language like Korean written in Hangul would need to create individual glyphs from smaller ones through th…

Why does Unicode need to represent a half-written Japanese word? If it's half-written, you're still in the process of writing it, and this is entirely the domain of your text input system. Which is to say, there is absolutely no need for the text input system to represent all stages of input as Unicode. It is free to represent the input however it chooses to do so, and only produce Unicode when each written unit is "…

>Why does Unicode need to represent a half-written Japanese word? If it's half-written, you're still in the process of writing it, and this is entirely the domain of your text input system.

Drafts or word documents (which are immensely simpler if just stored as unicode). Then there's the fact that people occasionally do funky things with kanji anyway, so you're doing everyone a favour by letting them half-write a word anyway.

Re: I Can’t Write My Name in Unicode

#259

Post by maxlybbert »

jimktrains2 wrote: post #257
maxlybbert wrote: post #244

Earlier quoted context omitted.

That's funny; I spent several hours of class time trilling r's to make sure we pronounced "carro" correctly, and repeating a 30 character alphabet.

rr not being its own letter has no bearing on if you can pronounce carro correctly, just like saying church right has no bearing on if c and h are two letters or ch is a single letter.

What can I say? Apparently the textbook was wrong.

Re: I Can’t Write My Name in Unicode

#260

Post by Qwertious »

ptaipale wrote: post #181
com2kid wrote: post #80

Earlier quoted context omitted.

Wikipedia is not forced unto the world as everyone's only source of knowledge. Unicode on the other hand is the only way many people have to input and see text in their native language. When one group proposes being the ultimate solution to everyone's problems, and then pushes their standard forward as such, complaints about inadequacies in the solution presented are perfectly fair and valid.

Unicode is not "forced" unto the world either. It's just a good way to do the thing it does, but it doesn't happen without contribution from those who are impacted.

The word is "unicode". "Uni" as in "united". Wikipedia doesn't claim to be the One True Encyclopedia, but with Unicode it's literally right there in the name.
Post reply on HN