Unicode Standard, Version 12.0 • Hacker News Board

Re: Unicode Standard, Version 12.0

#41

Post by vardump » Wed, Mar 06, 2019, 1:26 PM UTC

781 wrote: post #36

lelf wrote: post #34

Every top comment is about Emoji… Law of triviality in action ;) Also in Version 12.0, the following Unicode Standard Annexes have notable modifications ⟨…⟩ UAX #14, Unicode Linebreaking Algorithm UAX #29, Unicode Text Segmentation UAX #31, Unicode Identifier and Pattern Syntax UAX #38, Unicode Han Database (Unihan) UAX #45, U-Source Ideographs

It's not triviality. How many people do you really think care about Elymaic script? Or about Nandinagari?

Why would you bring up something relatively obscure and ignore all the other major non-emoji changes?

In addition to what grandparent listed:

> UTS #10, Unicode Collation Algorithm—sorting Unicode text

> UTS #39, Unicode Security Mechanisms—reducing Unicode spoofing

> UTS #46, Unicode IDNA Compatibility Processing—compatible processing of non-ASCII URLs

I think plenty of people care about these changes.

Re: Unicode Standard, Version 12.0

#42

Post by tialaramex » Wed, Mar 06, 2019, 1:27 PM UTC

wongarsu wrote: post #35

josteink wrote: post #32

Earlier quoted context omitted.

> Unicode is to support all scripts. Unicode is meant to make all other text encodings obsolete so the world never has to think about text encodings again Technically speaking Unicode is not an encoding, but otherwise your point is mostly correct.

I guess UTF-8 is technically what we would call the encoding (with alternatives like UTF-32 with other tradeoffs). But what would be the correct word for Unicode, if not encoding? I guess I could always say Unicode standard, but that feels like just avoiding the issue (for example we usually say SMTP protocol, not SMTP standard).

"Character Set" is usually the phrase.

A character set can be encoded in a variety of ways, for Unicode / ISO-10646 the encoding UTF-8 is the most popular for a variety of reasons that I'm sure will one day be an exciting historical artefact for HN readers to remark upon.

I don't like the word character, because it tends to cause idiots to build software that thinks Unicode codepoints are the indivisible unit out of which strings are made, and that's no more true than for bytes. I prefer the nice fuzzy word "squiggle" when I mean the thing you as a human are perhaps imagining when saying "character" and to use nice technical terms like "pictogram", "grapheme", "glyph", "code point", "code unit", "symbol", and so on when I mean those specific technical things. But in the phrase "character set" that's what we ended up with, so be it.

Re: Unicode Standard, Version 12.0

#43

Post by jameshart » Wed, Mar 06, 2019, 1:33 PM UTC

781 wrote: post #36

lelf wrote: post #34

Every top comment is about Emoji… Law of triviality in action ;) Also in Version 12.0, the following Unicode Standard Annexes have notable modifications ⟨…⟩ UAX #14, Unicode Linebreaking Algorithm UAX #29, Unicode Text Segmentation UAX #31, Unicode Identifier and Pattern Syntax UAX #38, Unicode Han Database (Unihan) UAX #45, U-Source Ideographs

It's not triviality. How many people do you really think care about Elymaic script? Or about Nandinagari?

Presumably some people. Assigning code points is the Unicode consortium’s job. That’s what unicode does. Nobody should be upset that they keep doing it.

But you ignore the substance of the parent’s post which is about the new elements of the Unicode standard which are not confined to assigning code points. There is substance there to be analyzed - there is material there about how Unicode should be used in defining identifier syntaxes which is of high relevance to an HN audience (eg in defining your new serverless framework, what characters should you allow in function names? Unicode now has a better answer for you than ‘[a-zA-Z_0-9-]’).

There are updates to Unicode security and idna support.

But no, sure, let’s complain about emoji and obscure languages.

Re: Unicode Standard, Version 12.0

#44

Post by nyuszika7h » Wed, Mar 06, 2019, 1:51 PM UTC

lelf wrote: post #34

Every top comment is about Emoji… Law of triviality in action ;) Also in Version 12.0, the following Unicode Standard Annexes have notable modifications ⟨…⟩ UAX #14, Unicode Linebreaking Algorithm UAX #29, Unicode Text Segmentation UAX #31, Unicode Identifier and Pattern Syntax UAX #38, Unicode Han Database (Unihan) UAX #45, U-Source Ideographs

Monospace is hard to read on mobile, please don't use it for quoting.

Re: Unicode Standard, Version 12.0

#45

Post by billpg » Wed, Mar 06, 2019, 2:16 PM UTC

Dylan16807 wrote: post #17

billpg wrote: post #11

Emojis are a plot to make English-speaking developers care about fixing their code to work with Unicode.

If only they hadn't tried to cut it down to 16 bits near the start, we could have avoided a lot of the partial support that emojis expose.

Ideally, we'd have started from UTF-8 and never even bothered with 16 bit chars in the first place.

Re: Unicode Standard, Version 12.0

#46

Post by sctb » Wed, Mar 06, 2019, 3:17 PM UTC

josteink wrote: post #2

I wonder what fringe diversity group has gotten their new custom emojis accepted this time. Does Unicode care about actual languages any more?

If you aren't going to post substantively, don't post at all. We're asked you several times before and eventually we ban accounts that won't follow the guidelines.

https://news.ycombinator.com/newsguidelines.html

Re: Unicode Standard, Version 12.0

#47

Post by rurban » Wed, Mar 06, 2019, 5:12 PM UTC

FWIW, I've updated now the safeclib to 12.0.0 final from the previous 12.0.0-d1, and there were no changes in the case folding tables. And the changes from 11.0 are minimal, just 6 new entries. So it's just a minimal libc specific update, thanksfully.

For utf8-safe languages there would be 4 new scripts to add, but this affects only rust, java and cperl. All others are unicode unsafe.

Re: Unicode Standard, Version 12.0

#48

Post by m6w6 » Wed, Mar 06, 2019, 6:05 PM UTC

magicalhippo wrote: post #22

thomasfedb wrote: post #14

Earlier quoted context omitted.

Emoji handling on Linux is so upsetting. It should not be this tricky.

I submit that it should: http://baldi.me/blog/emoji-in-sql

Hilarious.

Re: Unicode Standard, Version 12.0

#49

Post by Dylan16807 » Wed, Mar 06, 2019, 7:32 PM UTC

ygra wrote: post #31

titanix2 wrote: post #30

Earlier quoted context omitted.

U+2800 etc. is for Braille, which is an alphabet. Nothing to do with bitmaps.

I think it was a joke referring to some people using Braille to display black and white images as text, e.g. https://loveeevee.github.io/Dots-Converter/retro.html

Yeah. Key to that use is that they didn't pick any specific patterns, they went with every possible combination of 8 dots being present/absent.

Re: Unicode Standard, Version 12.0

#50

Post by Dylan16807 » Wed, Mar 06, 2019, 7:59 PM UTC

nikbackm wrote: post #40

Dylan16807 wrote: post #17

Earlier quoted context omitted.

If only they hadn't tried to cut it down to 16 bits near the start, we could have avoided a lot of the partial support that emojis expose.

Maybe all Asian scripts was not planned to be included back then? Seems strange they would miscount so grossly otherwise.

It's all down to CJK. Originally they allocated 21k codepoints to CJK, and if that was accurate then 16 bits would pretty much fit things.

But we currently have 88k CJK characters assigned out of possibly more than 100k total.

I can't easily find anything about how this went wrong and they got such a small number.