Live data from Hacker News

What every software developer must know about Unicode in 2023

tonsky.me

501–510 of 572 posts

Re: What every software developer must know about Unicode in 2023

#501

> Hell, you can’t even spell café, piñata, or naïve without Unicode. I must have missed something. All of those symbols are present in Extended ASCII (i.e. 8-bit).

Extended ascii is a bodge, and requires you to set a code page to pick the right set of "extended" characters. Unicode is also a superset of ascii though, so that sentence is right, on a technicality.

Re: What every software developer must know about Unicode in 2023

#502
post #477

Earlier quoted context omitted.

> These user-perceived characters are approximated by what is called a grapheme cluster, which can be determined programmatically. From everything i've read or heard about unicode, "determined programmatically" is false?

I quoted Unicode specification, so no, it's not false.

Well I can find 30 comments stating that it's false for all practical purposes only in this HN thread. From people who have more experience with Unicode than me.

Re: What every software developer must know about Unicode in 2023

#503
post #467
post #272

> The only modern language that gets it right is Swift: print("...".count) // => 1 And Erlang/Elixir! I guess they are not "cool" enough. But they correctly interpret that as one grapheme cluster. % erl +pc unicode > string:length("..."). 1 (... here is the U+1F926 U+1F3FB U+200D U+2642 U+FE0F emoji)

The author does refer to Elixir further down: > UPD: Erlang/Elixir seem to be doing the right thing, too.

Updated a day after it was mentioned here :-)

https://github.com/tonsky/tonsky.me/commit/5fbbb373025be3758...

Re: What every software developer must know about Unicode in 2023

#504

Earlier quoted context omitted.

The problem is not that you need character composition for some writing systems. It's that there are no rules that would help with everything having an unique representation internally. Even "put the code points forming the composed character in descending numerical order" would be better than nothing. If it was there from the start. However, the Unicode commitee is too busy adding new emojis to make their standard s…

There are rules for that, Unicode has standards (not only formal, but easily usable in most software libraries) for canonical forms that will collapse all the variations to a single representation. But, of course, unicode can't define that the standard will cover only the canonical forms, and couldn't do that since the start, as it needed backwards compatibility with various pre-unicode encodings which had mutually i…

> for canonical formS

There' your problem right there. Plural formS. It's not canonical if there are more of it.

Re: What every software developer must know about Unicode in 2023

#505
post #152

There's one part of this document that I would push extremely hard against, and that's the notion that "extended grapheme clusters" are the one true, right way to think of characters in Unicode, and therefore any language that views the length in any other way is doing it wrong. The truth of the matter is that there are several different definitions of "character", depending on what you want to use it for. An extende…

> An extended grapheme cluster is largely defined on "this visually displays as a single unit", which isn't necessarily correct for things like "display size in a monospace font" or "thing that gets deleted when you hit backspace." I'm sorry, but I fail to see how "This visually displays as a single unit" could ever differ from "Display size in a monospace font" or "Thing that gets deleted when you hit backspace".

If I have a text with niqqudim I am going to want to think of the niqqudim differently when editing despite the fact they are entwined with the consonants.

Re: What every software developer must know about Unicode in 2023

#506
post #325

Earlier quoted context omitted.

It is some time ago since I last used it, but I found that too many websites that I want to read require Javascript to even show you the main body of text, or a reasonable layout. Is that different now?

No. Just whitelist the main domain for sites that are obviously broken. Then try one or two likely subdomains if that's not enough. In the rare cases where it still wants more crap enabled, then it's usually not worth the effort, close tab and move on to something else. As you build up a whitelist over time, it becomes pretty rare that you need to interact with it more than a couple times per day. Yeah, it takes some…

I already have that routine with uBlock Origin. I don't think NoScript offers all of uBO's functionality, and I certainly won't do the same dance for two extensions, but I'll look into uBO's abilities to specifically block JS.

Re: What every software developer must know about Unicode in 2023

#507
post #160
post #136

Earlier quoted context omitted.

Can you write them with iOS keyboard? Or when you say Apple Notes and chat apps you just mean from desktop? Edit ①: seems the answer is not with the default iOS keyboard, but possible to paste it and perhaps possible with a third party keyboard that I'm not keen on trying (unless I hear of a keyboard that's both genuinely useful / better than default, and that doesn't send keystrokes to the developer - though I can't…

Speaking of third party keyboards, I’m still upset about what happened to Nintype[0]. I’ve never ever been able to type faster on mobile than with it’s intuitive hybrid input style of sliding and tapping, paired with AI that was actually good. It used to be quite performant, fully customizable, and it worked beautifully as a replacement for default on jailbroken iOS. Today, it’s buggy $5 abandonware that only makes m…

I just want to say thank you for introducing me to Keyboard 71. I've never heard of Nintype, but this thing is incredible!

Re: What every software developer must know about Unicode in 2023

#508
post #22

Is it just me, or is anyone else seeing what looks like the mouse pointer of everyone else reading the page, like 1,000 little ants on the screen

turned off javascript as soon as I saw it. Like trying to read with twenty mosquitos in your face.

If you're using firefox, toggling reader view should do the trick.

Re: What every software developer must know about Unicode in 2023

#510

Anyone else hate titles like this? There are millions of developers working on a large variety of things. It sounds so arrogant to me.

It references previous famous blogposts. Much like "$THING considered harmful" titles. You'd also have a hard time working with computers in the modern day without running into unicode.
Post reply on HN