Can there be overlaps between fonts in the private use area?
What every software developer must know about Unicode in 2023
121–130 of 572 posts
Re: What every software developer must know about Unicode in 2023
#122Is it just me, or is anyone else seeing what looks like the mouse pointer of everyone else reading the page, like 1,000 little ants on the screen
Good times. If you click on the sun switch the entire UI gets zeroed out and you get to use on:hover mouse shtick to read the UI through a fuzzy radius. Is Yoko Ono designing websites now?
Re: What every software developer must know about Unicode in 2023
#123> People are not limited to a single locale. For example, I can read and write English (USA), English (UK), German, and Russian. Which locale should I set my computer to? Ideally - the "English-World" locale is supposedly meant for us, cosmopolitans. It's included with Windows 10 and 11. Practically, as "English-World" was not available in the past (and still wasn't available on platforms other than Windows the last…
> English (USA), English (UK)
> This deserves a separate discussion. I doubt many English speakers (let alone those who don't live in a particular anglophone country) care to distinguish between English dialects.
While that is generally (though not always) true, I would assume it's really a stand in for the much more relevant zh locales.
It is also rather relevant to es locales (america spanish has diverged quite a bit from europe spanish hence the creation of es-419), definitely french (canadian french, to a lesser extend belgian and swiss), and german (because swiss german). And it might be relevant for ko if north korea ever stops being what it is.
Re: What every software developer must know about Unicode in 2023
#124Unicode is a total mess. In a sane system, "extended grapheme clusters" would equal "codepoints" and it wouldn't make a difference for 99% of languages. Now we ended up with grapheme clusters, normalization, decomposition, composition, Zalgo text, etc. But instead of deprecating this nonsense, Unicode doubled down with composed Emojis.
The writing systems were already like this when we got them. Unicode's "total mess" mostly just reflects that. Of course it would be convenient for you, the programmer, if the users wanted the software to do whatever was easiest for you, but obviously they want what's easiest for them, not you.
How is it easiest "for them" to have the mess instead of having the newer standard be less messy?
Re: What every software developer must know about Unicode in 2023
#125The mouse cursos ir really annoying, stopped reading for that reason
Re: What every software developer must know about Unicode in 2023
#126Earlier quoted context omitted.
Swift string type has got many different views, like UTF-8, UTF-16, Unicode Scalar, etc… so if you want to count the bytes or cut over a specific byte you still can.
that's not the issue defaults matter as in they should things you can just use by-default without thinking about it as swift is deeply rooted in UI design having a default of glyphs make sense and as rust is deeply rooted in unix server and system programming utf-8 bytes make a lot of sense through the moment your language becomes more general purpose you could argue having a default in any way is wrong and it should…
> as in they should things you can just use by-default without thinking about it
That time has passed. If you want to know the length of a string, you really should indicate what length type you mean.
Re: What every software developer must know about Unicode in 2023
#127Can there be overlaps between fonts in the private use area?
Yes. "Private" in this case means that you can't expect consistent behavior from one system to the next.
Re: What every software developer must know about Unicode in 2023
#128 > The only modern language that gets it right is Swift:
julia> using Unicode
# for some reason HN doesn't allow emoji
julia> graphemes(" ")
length-1 GraphemeIterator{String} for " "
help?> graphemes
search: graphemes
graphemes(s::AbstractString) -> GraphemeIterator
Return an iterator over substrings of s that correspond to the extended graphemes in the string, as defined by Unicode UAX #29. (Roughly, these are what users would perceive as single characters, even though they may contain more than one codepoint; for example a letter combined
with an accent mark is a single grapheme.)Re: What every software developer must know about Unicode in 2023
#129> People are not limited to a single locale. For example, I can read and write English (USA), English (UK), German, and Russian. Which locale should I set my computer to? Ideally - the "English-World" locale is supposedly meant for us, cosmopolitans. It's included with Windows 10 and 11. Practically, as "English-World" was not available in the past (and still wasn't available on platforms other than Windows the last…
As an Irish person, while we have en_IE which is great (and solves most of the problems you list re: Euro-centric defaults + English), I'd still quite like to have an even more broad / trans-language / "cosmopolitan" locale to use.
I mainly type in English but occasionally other languages - I use a combination of Mac & Linux - macOS has an (off-by-default but enable-able) lang-changer icon in the tray that is handy enough, but still annoying to have to toggle. Linux is much worse.
Mac also has quite a nice long-press-to-select-special character that at least makes for accessible (if not efficient) typing in multiple languages while using an English locale. Mobile keyboards pioneered this (& Android's current one even does simultanous multi-lang autocomplete, though it severely hurts accuracy).
---
> I doubt many English speakers care to distinguish between English dialects.
I think you'll find the opposite to be true. US English spellings & conventions are quite a departure from other dialects, so typing fluidly & naturally in any non-US dialect is going to net you a world of autocorrect pain in en_US. To the extent it renders many potentially essential spelling & grammar checkers completely unusable.
Re: What every software developer must know about Unicode in 2023
#130Unicode is a total mess. In a sane system, "extended grapheme clusters" would equal "codepoints" and it wouldn't make a difference for 99% of languages. Now we ended up with grapheme clusters, normalization, decomposition, composition, Zalgo text, etc. But instead of deprecating this nonsense, Unicode doubled down with composed Emojis.
I feel its the same as with any long standing computer system we have today. It was designed as more and more of the world came online and all the growing pains it came with. Could it be built from scratch today better? Yes. Will it? No. I suspect it will be around long after we are all dead. Same with IPv4 :V
For most software it doesn't really matter either.
I've written unicode-aware software for over a decade, doing a wide variety of programs, and I've never had to bother with all that mess.
If I'm parsing strings I'm looking for stuff in the 7-bit ASCII range which maps neatly onto the Unicode representations, and so I just need to take care to preserve the rest.
The only trouble I've had is that a lot of programmers haven't learned, or don't get, that text encoding is a thing and that it needs to be handled.
So they'll hand me an XML they claim is UTF-8 encoded, except that XML header was just copypasta and the actual XML document is encoded in some other system encoding like Windows-1252. Or worse, a mix of both.