How a comment on Hacker News led to 4½ new Unicode characters • Hacker News Board

Re: How a comment on Hacker News led to 4½ new Unicode characters

#331

Post by thaumasiotes » Thu, Jun 23, 2016, 5:39 PM UTC

msbarnett wrote: post #319

thaumasiotes wrote: post #318

Earlier quoted context omitted.

Unification is a mistake. But 囍 has nothing to do with unification. Do you think that 福倒 should have a code point? Would it be considered one of the chinese characters (very iffy) or one of the holiday symbols? The 天书 ( https://en.wikipedia.org/wiki/A_Book_from_the_Sky ), by design, consists solely of chinese characters that don't exist. (Theoretically. A couple of them, by oversight, did exist.) They are still recog…

> Unification is a mistake. But 囍 has nothing to do with unification. I'm not saying it does, I'm saying Unification illustrates the fact that the Consortium's decision-making with respect to CJK has changed over time, has frequently been illogical, and shouldn't be pointed at as an example of anything good or sane or worthy of precedent. The fact that 囍 has a code point but 福倒 doesn't have a codepoint is another exa…

>> Do you think that 福倒 should have a code point?

> Yes. If we want to be able to talk about it in text (like now), I want to be able to encode it in a standardized way.

This doesn't make any sense. We talk about things in text by using words, not direct representations. A dog emoji is not necessary or desirable for discussing dogs in text, and a 福倒 emoji is not necessary or desirable for discussing 福倒s in text.

Should the wikipedia page https://en.wikipedia.org/wiki/Statue_of_Liberty be edited to replace the cumbersome phrase "statue of liberty" with the more modern and convenient U+1F5FD 'STATUE OF LIBERTY'?

Re: How a comment on Hacker News led to 4½ new Unicode characters

#332

Post by leni536 » Thu, Jun 23, 2016, 5:39 PM UTC

yk wrote: post #310

leni536 wrote: post #63

Earlier quoted context omitted.

> The Unicode tables include examples for all graphical code points: http://unicode.org/charts/ . If you really wanted you can make them into a font No, last time I checked you are not legally allowed to do that.

I checked your link, then I proceeded to the terms of service and IANAL there they claim that all 'unicode software' is basically MIT licensed. (Please check with a lawyer before you conclude that word is MIT licensed.)

Now check any random chart:

    Fonts
    The shapes of the reference glyphs used in these code charts are not prescriptive. Considerable variation is to be
    expected in actual fonts. The particular fonts used in these charts were provided to the Unicode Consortium by a number
    of different font designers, who own the rights to the fonts.
    See http://www.unicode.org/charts/fonts.html for a list.
    
    Terms of Use
    You may freely use these code charts for personal or internal business uses only. You may not incorporate them either
    wholly or in part into any product or publication, or otherwise distribute them without express written permission from
    the Unicode Consortium. However, you may provide links to these charts.
    The fonts and font data used in production of these code charts may NOT be extracted, or used in any other way in any
    product or publication, without permission or license granted by the typeface owner(s).
    The Unicode Consortium is not liable for errors or omissions in this file or the standard itself. Information on characters
    added to the Unicode Standard since the publication of the most recent version of the Unicode Standard, as well as on
    characters currently being considered for addition to the Unicode Standard can be found on the Unicode web site.

Now you can try to track down the actual typeface owners one by one (this alone seems hopeless) and even then I doubt that you can get permission from all of them.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#333

Post by baby » Thu, Jun 23, 2016, 5:41 PM UTC

I was wondering why they would have snowmen in the language. And then it occurred to me that maybe, since the unicode set has so much room for characters, that they were planning to allow cross-language communication through emoticons.

Think about it, if you can represent anything human with emoticons. Then you can communicate through emoticons only! Maybe that's what the ancient Egyptians were hopping for?

Re: How a comment on Hacker News led to 4½ new Unicode characters

#334

Post by lilyball » Thu, Jun 23, 2016, 5:45 PM UTC

etatoby wrote: post #233

jessriedel wrote: post #224

Earlier quoted context omitted.

> Unicode enabled all sorts of cool things in computing that simply were not possible before or only with awful hacks and workarounds. Could you give an example? I don't know anything about this stuff.

A text file, a webpage, or a database table can only contain textual data in a given encoding. That's because every byte stored in the file, for example byte number 188, either means "¼" (as it does in ISO/IEC 8859-1, aka. Latin-1 or ANSI), or it means "ỳ" (as in ISO/IEC 8859-14) or "ｼ" (in JIS X 0201, one of the many Japanese encodings that were devised over the years.) How do you know which encoding a certain file…

Fun fact: Japanese even has a word for foreign characters, "gaiji", and it's extremely common in Japanese ePUBs to use small square images very frequently for characters not in the current font, using the term "gaiji" in the CSS class names used for these characters. And at least one mainstream ePUB reader has special code to detect these gaiji and adjust its rendering to make them behave better.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#335

Post by jshevek » Thu, Jun 23, 2016, 5:48 PM UTC

eridius wrote: post #325

jshevek wrote: post #322

Earlier quoted context omitted.

That's a shame. Its one thing to have absolute, iron fisted control over your own platform - its another to intentionally seek to limit people's self expression on other platforms by influencing the standard in this way.

Are you saying that a company with voting rights shouldn't be allowed to have influence on what goes into Unicode? That doesn't make any sense. Also, if you read the article, Apple wasn't the only party in favor of nixing the emoji.

> Are you saying that a company with voting rights shouldn't be allowed to have influence on what goes into Unicode?

Obviously not.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#336

Post by gpvos » Thu, Jun 23, 2016, 5:49 PM UTC

WalterBright wrote: post #71

I always wondered what was wrong with the glyph "ON" to denote on.

That's not a glyph, but an English word. It would be perfectly appropriate in English-speaking countries. Not so much in the rest of the world.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#337

Post by gpvos » Thu, Jun 23, 2016, 5:51 PM UTC

prima-facie wrote: post #99

Legitimate question: Why is Unicode littered with all those useless symbols? I can see the reasoning behind the standard (or very common) symbols or things like emoji, but having every possible glyph in UTF8 seems like a horrible waste. What if we want to add new glyphs in the next 10 years for emerging standards?

Which ones do you find useless? And why do you find them less useful than those, in my eyes, extremely useless emojis? I'd say a power on and power off symbol are pretty useful things to have in a font.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#338

Post by msbarnett » Thu, Jun 23, 2016, 5:53 PM UTC

thaumasiotes wrote: post #331

msbarnett wrote: post #319

Earlier quoted context omitted.

> Unification is a mistake. But 囍 has nothing to do with unification. I'm not saying it does, I'm saying Unification illustrates the fact that the Consortium's decision-making with respect to CJK has changed over time, has frequently been illogical, and shouldn't be pointed at as an example of anything good or sane or worthy of precedent. The fact that 囍 has a code point but 福倒 doesn't have a codepoint is another exa…

>> Do you think that 福倒 should have a code point? > Yes. If we want to be able to talk about it in text (like now), I want to be able to encode it in a standardized way. This doesn't make any sense. We talk about things in text by using words, not direct representations. A dog emoji is not necessary or desirable for discussing dogs in text, and a 福倒 emoji is not necessary or desirable for discussing 福倒s in text. Shou…

> A dog emoji is not necessary or desirable for discussing dogs in text, and a 福倒 emoji is not necessary or desirable for discussing 福倒s in text.

"Necessary" is an ill-defined and reductive way of looking at communication. History has shown us that you can't draw bright lines between things you, in the abstract, have decided are the "necessary" subset, and expect the world to follow along.

Linguists have come to understand that you can only describe and follow human, behaviour, not prescribe it.

Anyways, humans plainly found it necessary to annotate their text messages with pictoral indicators of their mood, to the point where it became so widely spread and such a mess that we felt it desirable to standardize the code-point representations. That it isn't desirable in all circumstances or appropriate in all registers of formality does not mean that it isn't an emergent behaviour which will continue to arise whether or not it is "necessary".

tl;dr I don't really give a shit that "dog emoji" isn't appropriate for an academic text on canine surgery. It's more than sufficient to me that it is used millions of times in text messages between regular human beings. Text needn't be formal text to deserve respect in encoding.

Re: How a comment on Hacker News led to 4½ new Unicode characters

#339

Post by Semiapies » Thu, Jun 23, 2016, 5:58 PM UTC

gwbas1c wrote: post #248

acz wrote: post #238

Earlier quoted context omitted.

Let’s start working on "SVG over UTF" RFC, should we?

Honestly, I think "SVG over UTF" makes a lot more sense. It's impossible to make a character set that supports every character known to man, because that just adds undue effort on every computer maker, ect, to keep up. So why don't we pick a very good set: perhaps every letter in every language in common use for the past 200 years? Then, for the oddball symbols that someone wants to mix in text, there can be some kin…

Because it's easier to throw in random icons than to actually accomplish the goal of "every letter in every language in common use for the past 200 years", or even "past 20 years".

Or, put another way:

'We have an unambiguous, cross-platform way to represent “PILE OF POO” (), while we’re still debating which of the 1.2 billion native Chinese speakers deserve to spell their own names correctly.'

https://modelviewculture.com/pieces/i-can-text-you-a-pile-of...

Re: How a comment on Hacker News led to 4½ new Unicode characters

#340

Post by aji » Thu, Jun 23, 2016, 6:00 PM UTC

copperx wrote: post #249

kens wrote: post #101

Earlier quoted context omitted.

Unicode is supposed to include symbols that appear in "running text", not standalone icons. So no on traffic signs for instance. (There are exceptions for historical reasons. And emoji are a totally separate story.)

I have never read a book that had a snowman in the running text, so what's the story for emoji?

Tom Scott has a great video about the history of emoji. Basically, some companies in far east countries were encoding these icons in various proprietary codes for their messaging systems. The ecosystem became widespread and consistent enough that the unicode consortium saw it fit to include these emoji in the standard.

The snowman, on the other hand, is a weather symbol for snow, I assume. It appears alongside other symbols for meteorological phenomena, so I imagine was added around the same time and with similar reasoning: http://www.fileformat.info/info/unicode/block/miscellaneous_...