Live data from Hacker News

What every software developer must know about Unicode in 2023

tonsky.me

361–370 of 572 posts

Re: What every software developer must know about Unicode in 2023

#361

Earlier quoted context omitted.

I've never even heard of it before.

That's a higher bar than having seen it, I think. I also had to look it up, but as soon as I saw the images in Wikipedia I knew that it's from Lord of the Rings.

It is. But the even higher bar is that you actually write in this script.

Re: What every software developer must know about Unicode in 2023

#362

Earlier quoted context omitted.

Honestly I like ipv4 better than v6. I like having a NAT and easy addresses like 192.168.1.3 instead of fe80::210:5aff:feaa:20a2. They didn't need to mess with those things just to expand the address space, like how utf8 didn't require remapping ASCII.

IPv4.1 should have just had 39 bits, to be written like 999.999.999.999. (I know this wouldn't have actually had much effect, nobody is going to add new routes in the middle of "class A" spaces that already existed, so it would just give those that already had IP addresses more IP addresses. Additionally, people really abuse decimal addresses in horrifying ways; for example, Fios steals 192.168.1.100-192.168.1.150 fo…

Having it actually be decimal might've been nice, but at this point people are used to the 1-254 range, and I think the least jarring addition of extra bits would be to simply extend it for the addresses that need them (and not for the ones that don't). So you could have 123.444.3.254 or longer like 123.444.3.254.12.43.

Re: What every software developer must know about Unicode in 2023

#363

There's one part of this document that I would push extremely hard against, and that's the notion that "extended grapheme clusters" are the one true, right way to think of characters in Unicode, and therefore any language that views the length in any other way is doing it wrong. The truth of the matter is that there are several different definitions of "character", depending on what you want to use it for. An extende…

String iteration should be based on whatever you want to iterate on - bytes, codepoints, grapheme clusters, words or paragraphs. There's no reason to privilege any one of these, and Swift doesn't do this. "Length" is a meaningless query because of this, but you might want to default to whatever approximates width in a UI label, so that's grapheme clusters. Using codepoints mostly means you wish you were doing bytes.

> There's no reason to privilege any one of these, and Swift doesn't do this.

Strange thing to say: Swift String count property is the count of extended grapheme clusters. The documentation is explicit:

> A string is a collection of extended grapheme clusters, which approximate human-readable characters. [emphasis in original]

Re: What every software developer must know about Unicode in 2023

#364

There's one part of this document that I would push extremely hard against, and that's the notion that "extended grapheme clusters" are the one true, right way to think of characters in Unicode, and therefore any language that views the length in any other way is doing it wrong. The truth of the matter is that there are several different definitions of "character", depending on what you want to use it for. An extende…

In that case, it sounds like `length` on Unicode strings simply shouldn't exist, since there is no obvious right answer for it. Instead there should be `codepointCount`, `graphemeCount`, etc.

Re: What every software developer must know about Unicode in 2023

#366

> For example, é (a single grapheme) is encoded in Unicode as e (U+0065 Latin Small Letter E) + ´ (U+0301 Combining Acute Accent). Two code points! It's a poor and misleading example for it is definitely not how 'é' is encoded in 99.999% of all the text written in, say, french out there (french is the language where 'é' is the most common). 'é' is U+00F9, one codepoint, definitely not two. Now you could say: but it…

> definitely not how 'é' is encoded in 99.999% of all the text written in, say, french out there

Maybe how it's input by the keyboard (I haven't checked) but not how it's output on the web or other documents.

Plenty of text goes through Unicode normalization which may convert it to two codepoints.

Re: What every software developer must know about Unicode in 2023

#367
post #321

Earlier quoted context omitted.

There are libraries that help with iterating both code-points and grapheme clusters... - but are there any of them that can help decide what to do for example when pressing backspace given an input string and a cursor position? Or any other text editing behavior. This use-case-dependent behavior must have some "correct" behavior that is standardized somewhere? Like a way to query what should be treated like a single…

> Or typing 'o' + '¨' to produce 'ö' but realizing you wanted to type 'ô', there just one backspace press would revert it to 'o' again and you could press '^' to get the 'ô'. This is a good example because in German I would expect 'o' + '¨' + to leave no character at all while in French I would expect 'e' + '`' + to leave the e behind because in my mind it was a typo. The rendering of brahmic- and arabic-derived scri…

In French, è is a single character issued by a single keypress on a French keyboard, like e, or +. (Note that A is shift+a). Why should it need two backspaces? If you press e+` well you have e`, not è.

Re: What every software developer must know about Unicode in 2023

#368
post #321

Earlier quoted context omitted.

> Or typing 'o' + '¨' to produce 'ö' but realizing you wanted to type 'ô', there just one backspace press would revert it to 'o' again and you could press '^' to get the 'ô'. This is a good example because in German I would expect 'o' + '¨' + to leave no character at all while in French I would expect 'e' + '`' + to leave the e behind because in my mind it was a typo. The rendering of brahmic- and arabic-derived scri…

But typing "ö" (e.g. swiss keyboard) and pressing delete & getting an o would be annoying af

Same for a French keyboard with éèàù which are all typed using one key. But even êôûæœäëïöü, all typed using at least two keys, if not 3 with a compose key (from memory, I'm using a phone). Everybody is used to the way it has been working on all OSes.

Re: What every software developer must know about Unicode in 2023

#369
post #34

Earlier quoted context omitted.

uBlock Origin -> Disable Javascript Problem solved!

That breaks the video. inspect -> network -> refresh -> blocking the request for pointers.js works.

It doesn't on Firefox, you get the built in media controls.
Post reply on HN