Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I Can’t Write My Name in Unicode

modelviewculture.com

Post reply on HN
Print view View original

151–160 of 377 posts

Re: I Can’t Write My Name in Unicode

#151

Post by microtherion »

_yosefk wrote: post #76
cplease wrote: post #69

Earlier quoted context omitted.

> Hebrew (and I'd guess Arabic and other right-to-left languages) work rather badly in Unicode when it comes to bidirectional rendering; however, to the extent that it's a result of Israeli/Egyptian/Saudi/etc. companies and/or governments failing to pay $18K (the figure from TFA) to pay for the consortium membership I'm sorry? How does this have anything to do with encoding, versus software toolkits? Unicode and enco…

Unicode defines a "logical order" and a "rendering order". The logical order is supposed to be the order in which you read the text - letters read earlier are closer to the beginning of the buffer. The rendering order is how the stuff appears on screen - where an English word inside Hebrew text, if you count the letters from right to left, will obviously have its last letter assigned a smaller number than its first l…

You could keep them reversed in the file

I don't think you could, if the screen rendering has to do any soft line breaking.

Yes, bidirectional rendering is messy, but I don't think it's fair to blame the mess on Unicode without a clearly superior alternative.

Re: I Can’t Write My Name in Unicode

#152

Post by com2kid »

Tomte wrote: post #135
austinz wrote: post #128

Earlier quoted context omitted.

The difference between traditional and simplified Chinese characters is more than simply different fonts. Part of the difficulty is that some simplified characters map to multiple traditional characters, which means that converting from one to the other may be lossy. There's also the Japanese equivalent of simplified characters (shinjitai), many of which differ from their Chinese counterparts, as well as characters t…

Some. I know that not every letter has been Han-unified, but I don't know the specifics. Is it possible, that this problem of yours has actually been taken care of? If not, is it possible that it's just a mistake instead of a big evil conspiracy?

AFAIK it was a historical attempt to save on encoding space back when Unicode had a 16bit fixed width and could only support up to 65K characters.

Now that Unicode has expanded out, I am confused why anyone still defends this practice.

Re: I Can’t Write My Name in Unicode

#153

Post by com2kid »

Tomte wrote: post #129
com2kid wrote: post #122

Earlier quoted context omitted.

Those are all stylistic differences. You can still read any of those fonts. Sure some designer can go overboard and make a font hard to read, but that is just a designer being overly fancy! Think instead of the difference between Cyrillic and Latin. Sure if you squint hard enough they both have a common origin (Greek), but you'd be rather annoyed if you setup your phone, selected "English" and the OS used Cyrillic le…

Actually, I can't. I can decipher quite a bit with lots of effort, but I wouldn't call that "reading". And there are very few people around who can read all those (especially Textura) without problems. On your other example: when I was in Russia, I found those "unknown" letters difficult, but fun. But the letters they share with Latin? I didn't see a difference. And to your phone example: of course, I'd be annoyed. B…

> On your other example: when I was in Russia, I found those "unknown" letters difficult, but fun. But the letters they share with Latin? I didn't see a difference.

Again, Han unification does this.

Most characters are the same, great! But some are different. Sucks for those that are different.

> And to your phone example: of course, I'd be annoyed. But that's exactly the point: you have to set up your local system correctly, to your expectations and standards.

The problem here is that for almost everything else, Unicode is a mapping from a code point (or a set of code points) to some distinct and unique representation on screen.

Except there are some code points for which that isn't true.

> Sorry, I truly think you've run up an argumentative dead end.

You are not arguing any point, other than "let's keep this historical attempt at saving on encoding space around, even though we have expanded out the encoding space so that we don't need the savings anymore."

That isn't a strong argument.

My argument is "user's don't like this, it upsets them, we shouldn't do it."

If something we do as engineers angers or upsets our users, we are doing it wrong. Flat out.

Re: I Can’t Write My Name in Unicode

#154

Post by ckoerner »

The comparison to the play "My Fair Lady" is not very convincing. I would suggest the author remove it as it weakens the argument. First, the fictional character states 'no fewer than' as an admittance that there are more. Second, even if we consider this as a valid complaint, the author himself points out in his example that this 'common sentiment' is from a play written a century ago. Using a fictional character from a time when the world's knowledge of language was incredibly smaller than it is today does not help support your goals.

Re: I Can’t Write My Name in Unicode

#155

Post by sdg1 »

chimeracoder wrote: post #134
sdg1 wrote: post #117

Earlier quoted context omitted.

> The Bengali fonts that you have installed happen to render it as a jophola It's not only the Bengali font - the text rendering framework of my operating system also needs to have a bunch of complex rules to figure out that a jophola needs to be rendered. It also needs to know that the visual ordering of i-kar is before the preceding consonant cluster (দ in আদিত্য). > the characters that are required to type a jopho…

Is Bengali your first language? While one can make the case that ত্য is simply "'to' - 'o' + 'ya' = 'to'"[0][1], it's rather confusing mental acrobatics, and it doesn't reflect either how the writing system is taught, or how native speakers use it and think of it on a day-to-day basis. If anything, your comment makes a stronger argument for consolidating ই and ি (they are literally the same letter and phoneme, but wr…

> Is Bengali your first language?

Yes.

> [...] it's rather confusing mental acrobatics, and it doesn't reflect either how the writing system is taught, or how native speakers use it and think of it on a day-to-day basis.

Mental acrobatics are part-and-parcel of the language, either in digital or non-digital form. If I were to spell out your name aloud, I would end with "ত-এ য-ফলা", which doesn't really say anything about how ত্য is pronounced. While writing on paper, we say "ক-এ ইকার", and then we reorder what we just said to write the ইকার in front of the ক. Even more complicated mental acrobatics - we say ক-এ ওকার, and then proceed to write half of the ওকার in front of the ক and then the other half, after the ক. We don't necessarily think about these when we carry out these acrobatics in our head, but they exist, and we have made the layer on top of the encoding system (rendering, and to some extent, input) deal with these acrobatics as well. My point in the original comment (and to some extent in the preceding one) was to emphasize that a lot of these issues are at the input method level - we should not have to think about encoding as long as it accurately and unambiguously represent whatever we want it to represent.

Just out of curiosity - I would be interested to know more about your learning experience that you feel is not well aligned with the representation of jophola as it is currently.

Re: I Can’t Write My Name in Unicode

#156

Post by vacri »

It sounds like the author is looking to be offended. Talking about Bengali being the seventh largest native language, then saying that US$18k is too expensive for a stake in solving this problem? Emojis with skin tones, something every human can use, that have arrived before a character that only Bengalis use is taken as an "outright insult"?

Re: I Can’t Write My Name in Unicode

#157

Post by Tomte »

com2kid wrote: post #152
Tomte wrote: post #135

Earlier quoted context omitted.

Some. I know that not every letter has been Han-unified, but I don't know the specifics. Is it possible, that this problem of yours has actually been taken care of? If not, is it possible that it's just a mistake instead of a big evil conspiracy?

AFAIK it was a historical attempt to save on encoding space back when Unicode had a 16bit fixed width and could only support up to 65K characters. Now that Unicode has expanded out, I am confused why anyone still defends this practice.

That's just one of the reasons people had (and you can't really expect everyone involved to have totally congruent reasons).

But how about "g"?

Do you really believe the two common variants (one storey/two stories) should have separate code points?

What about German vs. French vs. Danish vs. etc.? All different "g"?

Why? And if not, what is the core difference?

Re: I Can’t Write My Name in Unicode

#158

Post by darklajid »

maxlybbert wrote: post #148
chimeracoder wrote: post #109

Earlier quoted context omitted.

> Unicode makes extensive use of combining characters for european languages, for example to produce diacritics: ìǒ or even for flag emoji. But it doesn't, for example say that a lowercase "b" is simply "a lowercase 'l' followed by an 'o' followed by an invisible joiner", because no native English speaker thinks of the character "b" as even remotely related to "lo" when reading and writing. > It seems like you're try…

> > It seems like you're trying to single out combining pairs as "less legitimate" when they're extensively used in the standard. > I'm saying that Unicode only does it in English where it makes semantic sense to a native English speaker. Well, combining characters almost never come up in English. The best I can think of would be the use of cedillas, diaereses, and acute accents in words like façade, coördinate and…

I'm coming from a German background and I sympathize with the author.

German has 4 (7 if you consider cases) non-ASCII characters: äüöß(and upper-case umlauts). All of these are unique, well-defined codepoints.

That's not related to composing on a keyboard. In fact, although I'm German I'm using the US keyboard layout and HAD to compose these characters now. But I wouldn't need to and the result is a single codepoint again..

Re: I Can’t Write My Name in Unicode

#159

Post by com2kid »

astrange wrote: post #138
com2kid wrote: post #89

Earlier quoted context omitted.

BINGO. But because of Han unification I all of a sudden DO need to know the language. The same Unicode code point needs to be rendered differently for a user in Mainland China versus a user in Japan or else the user may not be able to read the text! Even if the user can read the character, they are going to experience a degradation in reading speed and comprehension, and be generally frustrated. Not to mention showin…

In what situations do you need to do this, but don't need to show any other data (dates and times, localized UI, user timezone, culturally appropriate fonts, RTLness) that involves knowing the user's languages and locale? This can happen if the user is intentionally reading mixed-language text or text not in their computer's UI language, of course. In that case different CJK languages also have different preferred fo…

> In what situations do you need to do this, but don't need to show any other data (dates and times, localized UI, user timezone, culturally appropriate fonts, RTLness) that involves knowing the user's languages and locale?

For drawing a given glyph, there is normally a lookup into a font table that involves solely the string of Unicode code points coming in.

Except if any characters in the CJK Unified Ideograph range. Then my function call suddenly has to jump out to read environment variables, which are hopefully setup correctly.

My code to do a lookup into a font file should not depend upon the users environment variables due to a space saving optimization made two decades ago.

Re: I Can’t Write My Name in Unicode

#160

Post by marincounty »

My English professor biggest pet peeve. He told us day one.

It was:

1. Don't use cliches in your writing. 2. Don't ever use emoji in any communication.

p.s. Emoji wasen't even around when I went to school. I bet Mr. Taylor woukd have had a field day though? Never been the fan of any smiley face, even rotated 90 degrees.

Post reply on HN