Live data from Hacker News

What every software developer must know about Unicode in 2023

tonsky.me

161–170 of 572 posts

Re: What every software developer must know about Unicode in 2023

#161

Earlier quoted context omitted.

I would wonder how many people are here who have never seen Tengwar. I would bet that's a minuscule minority.

I've never even heard of it before.

That's a higher bar than having seen it, I think. I also had to look it up, but as soon as I saw the images in Wikipedia I knew that it's from Lord of the Rings.

Re: What every software developer must know about Unicode in 2023

#162
> The minimum every software developer must know about Unicode

Just a nitpick...

Once more, as it is typical on HN, web programming is confused with the entire universe of software development.

There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough.

Re: What every software developer must know about Unicode in 2023

#165

Prior to this article, I knew graphemes were a thing and that proper unicode software is supposed to count those instead of bytes or code points. I didn't know that unicode changes the definition of grapheme in backwards incompatible fashion annually, so software which works by grapheme count is probably inconsistent with other software using a different version of the standard anyway. I'm therefore going to continue…

Two Unicode strings can be visually and semantically identical, but not byte-equal.

Re: What every software developer must know about Unicode in 2023

#166
post #128

> The only modern language that gets it right is Swift: arguably not true: julia> using Unicode # for some reason HN doesn't allow emoji julia> graphemes(" ") length-1 GraphemeIterator{String} for " " help?> graphemes search: graphemes graphemes(s::AbstractString) -> GraphemeIterator Return an iterator over substrings of s that correspond to the extended graphemes in the string, as defined by Unicode UAX #29. (Roughl…

Raku also gets it right.

Re: What every software developer must know about Unicode in 2023

#167

Unicode is a total mess. In a sane system, "extended grapheme clusters" would equal "codepoints" and it wouldn't make a difference for 99% of languages. Now we ended up with grapheme clusters, normalization, decomposition, composition, Zalgo text, etc. But instead of deprecating this nonsense, Unicode doubled down with composed Emojis.

I feel its the same as with any long standing computer system we have today. It was designed as more and more of the world came online and all the growing pains it came with. Could it be built from scratch today better? Yes. Will it? No. I suspect it will be around long after we are all dead. Same with IPv4 :V

Honestly I like ipv4 better than v6. I like having a NAT and easy addresses like 192.168.1.3 instead of fe80::210:5aff:feaa:20a2. They didn't need to mess with those things just to expand the address space, like how utf8 didn't require remapping ASCII.

Re: What every software developer must know about Unicode in 2023

#168

Is it just me, or is anyone else seeing what looks like the mouse pointer of everyone else reading the page, like 1,000 little ants on the screen

Anytime tonsky's site gets posted here, I'm reminded by how awful it is, which is ironic given his UI/UX background. The site's lightmode is a blinding saturated yellow, and if you switch into darkmode, it's an even less readable "cute" flashlight js trick. I don't know why he thought this was a good idea. Thank god for Firefox reader mode.

Works like a normal website with JavaScript disabled. I didn't even know it did fancy junk until reading the comments here. NoScript saves the day again! I don't know how people can browse the web without it.

Re: What every software developer must know about Unicode in 2023

#169
post #162

> The minimum every software developer must know about Unicode Just a nitpick... Once more, as it is typical on HN, web programming is confused with the entire universe of software development. There are plenty of software realms where ASCII not only is enough, but it actually MUST be enough.

What do you mean by “must be enough”?

Not being able to support non-latin scripts sounds more like a limitation than a feature to me, although of course in many contexts it’s not in any individual organizations power to overcome it.

Post reply on HN