Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a comment on Hacker News led to 4½ new Unicode characters

unicodepowersymbol.com

Post reply on HN
Print view View original

291–300 of 429 posts

Re: How a comment on Hacker News led to 4½ new Unicode characters

#291

Post by acdha »

Semiapies wrote: post #289
acdha wrote: post #283

Earlier quoted context omitted.

When was the last time you checked? See e.g. http://www.unicode.org/charts/ and especially http://unicode.org/alloc/Pipeline.html – it's not everything in human history but what's left out are increasingly obscure.

You don't have to point to human history, though that's a good source of missing scripts. Waving off scripts actually in use as "increasingly obscure", while cheering Unicode throwing in any icon random geeks pitch to them, misses the purpose of Unicode.

You're tossing that assertion around without supporting it – what commonly used characters are not in Unicode? How many people use them? Are they not in Unicode because nobody cares or because there is a lack of someone authoritative helping codify the list or contentious disagreements about some aspects of that work?

Re: How a comment on Hacker News led to 4½ new Unicode characters

#292

Post by macintux »

pron wrote: post #221
johncolanduoni wrote: post #186

Earlier quoted context omitted.

Unicode has added 1791 emojis[1]. Note that it has used significantly fewer codepoints than this for emojis, because some (e.g. flags) are done via a small number of combining marks (flags use 26 code points for letters in country codes). They've been slowing down the rate of additional emojis since they started doing this. Do you really think they'll accelerate at some point and use up the almost 1 million unassigne…

I'm not afraid of running out of codepoints. I just think it is misguided that non-text is standardized as text. It just doesn't make sense. I don't know how people will communicate 50 years from now, but I think it's funny that even then, text strings would still need to support all those vegetables and hand gestures. Written text is pretty much eternal; emojis and pictograms? I doubt it. It doesn't make sense for t…

On a long-enough time scale, emojis and pictograms is what we're relying on[1] for nuclear waste warnings.

1: http://large.stanford.edu/courses/2011/ph241/dunn2/

Re: How a comment on Hacker News led to 4½ new Unicode characters

#293

Post by thaumasiotes »

ximeng wrote: post #274
thaumasiotes wrote: post #256

Earlier quoted context omitted.

Technically, glyphs are supposed to meet some standards, like being shown in use in running text, before they can be added to unicode. It's not supposed to be a repository of every picture anyone ever dreamed up. The standards are not applied consistently. Even leaving emoji out of it, the chinese "character" 囍 never occurs in running text, but there it is in unicode.

I don't think it's true that 囍 never occurs in running text - it's used in company names which would be used in text. It would be odd not to have an encoding for such a common character.

All right, I spent some time trying to find the requirement. I did not find it, but my tentative conclusion is that it does not apply to chinese characters.

FROM MEMORY, a while back there was an article on HN complaining that emoji seemed to magically bypass the requirements other characters needed to meet for inclusion in unicode, and that in fact they were commonly in violation. The taco symbol was called out as an example. I can no longer find this article, but it mentioned the running text requirement, and -- I believe -- specifically indicated that use in names does not count as use in running text. (For an idea of why that might be the case, check out http://tvtropes.org/pmwiki/pmwiki.php/Main/LuckyCharmsTitle .)

HOWEVER, I was not even able to find, on the unicode web site, any discussion of a running text requirement at all, for any kind of symbol. Some example proposals do refer to "running text" by name, but they don't indicate why. The example proposal given for adding characters to an existing block ( http://std.dkuug.dk/jtc1/sc2/wg2/docs/n2934.pdf , suggested as a prototype in http://www.unicode.org/faq/char_proposal.html ) does not mention "running text" at all, and doesn't appear to go to much trouble to document it, although some such documentation is given. The rough guidelines for character proposals at http://unicode.org/pending/proposals.html do not refer to "running text" at all, but they do suggest that, late in the process (specifically, on a proposal summary form, which is different from, and subsequent to, an actual proposal), "references to dictionaries and descriptive texts establishing authoritative information" are required.

I conclude that the Unicode standard's preferred criterion for chinese character inclusion is "would an authoritative chinese dictionary include this character", and while the answer to that question for 囍 is not unambiguous -- a lot of dictionaries don't include it -- it's easy to imagine that some do.

I would appreciate a pointer to the actual running text requirements, as well as what they are supposed to apply to, if anyone can provide that.

> It would be odd not to have an encoding for such a common character.

Outside of its use as a wedding decoration, which is plainly nonlinguistic, how common is it?

Re: How a comment on Hacker News led to 4½ new Unicode characters

#294

Post by thaumasiotes »

footpath wrote: post #276
thaumasiotes wrote: post #265

Earlier quoted context omitted.

> For CJK characters, they unified all semantically similar han-characters, even when they have visual forms that are quite different between Japanese, Chinese and Korean. This isn't true. 青 and 靑 are the same character written differently; they have their own codepoints. Ditto for a huge number of simplified Chinese characters; 语 is mainland Chinese and 語 is the same character in Japanese.

It is true to some extent. While 青 and 靑 have different codepoints, there are plenty of characters of the same codepoint that are rendered differently depends on the language specificed: https://en.wikipedia.org/wiki/Han_unification#Examples_of_la... Han characters that are traditionally viewed as variants of one another, or that are simplified from more complex logograms (such as 龜, which was simplified into 亀 in Ja…

I do know about the issue; it causes problems for me. But I couldn't let the claim that all semantically equivalent characters were unified pass.

> the stylistically different ones usually belong to the same codepoint

Fair enough. Do you happen to know why 青 and 靑 weren't unified?

Re: How a comment on Hacker News led to 4½ new Unicode characters

#295

Post by barkingcat »

thaumasiotes wrote: post #256
wmil wrote: post #241

Earlier quoted context omitted.

They want every symbol used in a document to have a unique encoding, so that you can change fonts without losing meaning. Fonts like wingdings are a horrible hack. The idea is one (complex) encoding that will represent the info until the end of time. It creates a lot of trouble, but it's still a good idea.

Technically, glyphs are supposed to meet some standards, like being shown in use in running text, before they can be added to unicode. It's not supposed to be a repository of every picture anyone ever dreamed up. The standards are not applied consistently. Even leaving emoji out of it, the chinese "character" 囍 never occurs in running text, but there it is in unicode.

That character is super common. Way more common than any emoji I'd expect (even teens using emojis won't outweigh the number of weddings and new years in Chinese speaking countries).

Re: How a comment on Hacker News led to 4½ new Unicode characters

#296

Post by barkingcat »

thaumasiotes wrote: post #293
ximeng wrote: post #274

Earlier quoted context omitted.

I don't think it's true that 囍 never occurs in running text - it's used in company names which would be used in text. It would be odd not to have an encoding for such a common character.

All right, I spent some time trying to find the requirement. I did not find it, but my tentative conclusion is that it does not apply to chinese characters. FROM MEMORY, a while back there was an article on HN complaining that emoji seemed to magically bypass the requirements other characters needed to meet for inclusion in unicode, and that in fact they were commonly in violation. The taco symbol was called out as a…

That character is extremely common. It's not just a one off wedding celebration thing.

Also used in new year celebrations. Basically every single year you'd see tons of these printed.

https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy)

It's a pun actually, double happiness.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#297

Post by thaumasiotes »

barkingcat wrote: post #295
thaumasiotes wrote: post #256

Earlier quoted context omitted.

Technically, glyphs are supposed to meet some standards, like being shown in use in running text, before they can be added to unicode. It's not supposed to be a repository of every picture anyone ever dreamed up. The standards are not applied consistently. Even leaving emoji out of it, the chinese "character" 囍 never occurs in running text, but there it is in unicode.

That character is super common. Way more common than any emoji I'd expect (even teens using emojis won't outweigh the number of weddings and new years in Chinese speaking countries).

The question was whether it can meet the Unicode inclusion requirements, not whether it's common.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#298

Post by barkingcat »

thaumasiotes wrote: post #297
barkingcat wrote: post #295

Earlier quoted context omitted.

That character is super common. Way more common than any emoji I'd expect (even teens using emojis won't outweigh the number of weddings and new years in Chinese speaking countries).

The question was whether it can meet the Unicode inclusion requirements, not whether it's common.

Well, even if it doesn't meet unicode inclusion requirements, it is necessary for printing in one of the largest markets in the world. Without that character in unicode, Chinese display systems and printers probably won't use unicode at all (and before unicode they used some standard of their own) - meaning the question is whether unicode wants to be relevant or not, not whether the inclusion requirements fit.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#299

Post by acdha »

ck2 wrote: post #287

Great work but the sheer amount of unicode is getting crazy. Starting to think it should all just be svg

How would search work with SVG? What about support for screen readers or translation? Having standard symbols like this means that all of those are easily solved without needing something like an image classifier.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#300

Post by thaumasiotes »

barkingcat wrote: post #296
thaumasiotes wrote: post #293

Earlier quoted context omitted.

All right, I spent some time trying to find the requirement. I did not find it, but my tentative conclusion is that it does not apply to chinese characters. FROM MEMORY, a while back there was an article on HN complaining that emoji seemed to magically bypass the requirements other characters needed to meet for inclusion in unicode, and that in fact they were commonly in violation. The taco symbol was called out as a…

That character is extremely common. It's not just a one off wedding celebration thing. Also used in new year celebrations. Basically every single year you'd see tons of these printed. https://en.wikipedia.org/wiki/Double_Happiness_(calligraphy) It's a pun actually, double happiness.

Every year you see tons of 福倒 printed too (福, but upside down). That has yet to receive a code point.
Post reply on HN