Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

221–230 of 377 posts

Re: I Can’t Write My Name in Unicode

#221

Post by zhemao »

toufka wrote: post #139
frivoal wrote: post #82

Earlier quoted context omitted.

I'm fluent in Japanese and speak some Mandarin Chinese as well. These 3 characters are identical, not similar. For a different example, 国 and 國 used to be the same character, but China and Japan (left) have both diverged the traditional form still used in Taiwan (right). Unicode treats them as separate. 今 Looks slightly different in traditional Chinese vs other languages. In traditional Chinese, the little straight l…

This seems again to be a perfect place for rendering rather than encoding. The english letter 'a' can be rendered as a ring with a tail (the way I handwrite), or a ring with a cap and a tail (the way the font usually renders). Both are the same letter, if rendered differently based on my (contextually sensitive) font.

Except you can still recognize the 'a' as 'a' no matter which way it is rendered.

Not so with Chinese characters. For instance, the character for "fly" in simplified (飞) and traditional (飛) look very different. Someone who only learned simplified may not recognize the traditional character as being the same.

Re: I Can’t Write My Name in Unicode

#222

Post by eridius »

stevejones wrote: post #190
PaulAJ wrote: post #133

Earlier quoted context omitted.

Its not an AI issue, just a small matter of having lots of rules. Moreover this is not just an issue for non-Western languages: the character â (lower case "a" with a circumflex) can be represented either as a single code-point U+00E2 or as an "a" combined with a "^". Furthermore Unicode implementations are required to evaluate these two versions as being equal in string comparisons, so if you search for the combined…

In some languages those accented characters would be different letters, sometimes appearing far away from each other in collation order. In other cases they are basically the same letter. Whereas in Hungarian 'dzs' is a letter.

Different languages can define different collation rules even when they use the same graphemes. For example, in Swedish z < ö, but in German ö < z. Same graphemes, different collation.

Re: I Can’t Write My Name in Unicode

#223

Post by zhemao »

cosarara97 wrote: post #28
theon144 wrote: post #15

I don't understand; I don't feel like character combination using the zero width joiner is on the same level as 13375p34k. It looks like the character just doesn't have a separate code-point, but is instead a composite, but still technically "reachable" from within Unicode, no?

It's like typing ` + o to get "ò", isn't it? You can argue that ò is actually an o with that tilde, while that character is not ত + ্ + an invisible joining character, but that's an input method thing, and there is a ৎ character after all.

But ò has it's own codepoint, whereas the ending character in the Aditya's name does not.

Re: I Can’t Write My Name in Unicode

#224

Post by 1ris »

mikeash wrote: post #119
mlmonkey wrote: post #21

Earlier quoted context omitted.

That's ridiculous. How can you expect UC to reach out to the 1000s of languages out there? Plus, shouldn't the party who expects to benefit put in the effort? Did you know that the Indian Government has a department for just such a thing, http://tdil.mit.gov.in/ ? What has it been doing all these years? Added later: there's also CDAC http://cdac.in/index.aspx?id=mlingual . And these are just 2 that I found quickly.

How can you not expect UC to reach out to the 1000s of languages out there, when their entire and sole reason to exist is to allow computers to work with all of those languages? There's no point in taking on a job and then declaring it to be too difficult.

UC's job is to coordinate. There are plenty initiatives for all kind of languages. I havn't contributed, but I followed the mediaeval unicode initiative a bit. The barriers for contributing seem very low. The Wikipedia-analogy elsewhere here describes the situation very well. Complaing Unicode sucks for language X and the consortium is to blame is like complain that the x language Wikipedia sucks and wikimedia is to blame.

Re: I Can’t Write My Name in Unicode

#225

Post by LeoPanthera »

This article is imbued with its own form of curious racism. In particular, I became suspicious of its motives at the line:

> "It took half a century to replace the English-only ASCII with Unicode, and even that was only made possible with an encoding that explicitly maintains compatibility with ASCII, allowing English speakers to continue ignoring other languages."

ASCII-compatibilty was essential to ensure the adoption of Unicode. It's not because English speakers wanted to ignore anyone or anything, it's because without it, it would never have been adopted, and we'd be in a much worse position today.

In other words, the explanation is technical, not racial.

Re: I Can’t Write My Name in Unicode

#227

Post by zhemao »

vacri wrote: post #156

It sounds like the author is looking to be offended. Talking about Bengali being the seventh largest native language, then saying that US$18k is too expensive for a stake in solving this problem? Emojis with skin tones, something every human can use, that have arrived before a character that only Bengalis use is taken as an " outright insult "?

> Talking about Bengali being the seventh largest native language, then saying that US$18k is too expensive for a stake in solving this problem?

West Bengal and Bangladesh aren't exactly the richest places in the world.

> Emojis with skin tones, something every human can use, that have arrived before a character that only Bengalis use is taken as an "outright insult"?

Nobody is being inconvenienced by the inability to send emojis with a darker skin tone. People definitely are being inconvenienced by not being able to write a common character of their native language.

Re: I Can’t Write My Name in Unicode

#228

Post by ubasu »

lilyball wrote: post #207
ubasu wrote: post #77

Earlier quoted context omitted.

It has to do with how the text is rendered. For example, if you see the Bengali text on page 3 of this PDF: http://www.unicode.org/L2/L2004/04252-khanda-ta-review.pdf it is unreadable and incorrect Bengali. ;-)

On page 3 I see two different pieces of Bengali. One is text, and the other is an image. I assume you're referring to the text? What makes it wrong? And what software are you using to view the PDF? If it's wrong, it's quite possible that the software you're using doesn't render it correctly, rather than the document actually being wrong.

I am referring to the first line of Bengali text.

I have viewed it in Acrobat Reader, epdfview, Chrome's PDF Reader, Firefox's pdf reader and my iPhone's pdf reader.

The problems are that joint-letter ligatures are not used, and several vowels signs are placed after the consonant when they should have been placed before them.

Re: I Can’t Write My Name in Unicode

#229

Post by lilyball »

ubasu wrote: post #228
lilyball wrote: post #207

Earlier quoted context omitted.

On page 3 I see two different pieces of Bengali. One is text, and the other is an image. I assume you're referring to the text? What makes it wrong? And what software are you using to view the PDF? If it's wrong, it's quite possible that the software you're using doesn't render it correctly, rather than the document actually being wrong.

I am referring to the first line of Bengali text. I have viewed it in Acrobat Reader, epdfview, Chrome's PDF Reader, Firefox's pdf reader and my iPhone's pdf reader. The problems are that joint-letter ligatures are not used, and several vowels signs are placed after the consonant when they should have been placed before them.

Your description makes it sound like the text in the PDF is merely incorrect, rather than there being any issue with Unicode.

Re: I Can’t Write My Name in Unicode

#230

Post by 1ris »

darklajid wrote: post #208
1ris wrote: post #194

Earlier quoted context omitted.

8 letters actually. 'ẞ' was added quite a while later.

Agreed, it exists. But then again, most systems in use today (as far as I'm aware) would turn a ß into SS, not ẞ. Actually I think I've never seen a ẞ in use, ever. Not once. Now I'm running around testing 'Try $programmingLanguage' services on the net. Try Clojure for example: > (.toUpperCase "ß") "SS"

In Haskell: isLower $ toUpper 'ß' is True. I wonder how many security holes this unexpected behaviour causes.
Post reply on HN