Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What every software developer must know about Unicode in 2023

tonsky.me

Post reply on HN
Print view View original

511–520 of 572 posts

Re: What every software developer must know about Unicode in 2023

#511

Post by extraduder_ire »

ssokolow wrote: post #431
redder23 wrote: post #415

If just for the fact that it annoys people I love the mouse cursor idea. But I also find it technically interesting. Is some kind of consent legally needed per GDPR or something for this? I for sure is tracking, literally. And a website has to ask to set cookies ...

Cookie consent is only necessary if you're sharing it with others (eg. ad networks, Google Analytics, etc.) or using it for "non-essential" functions (again, stuff like analytics). Sites just don't want the general public to realize that. As for the mouse cursors, I don't think they qualify as personal information under the GDPR, but IANAL.

It's needed when you collect the data. Cursor position is being used directly to make the service "function" though. Even if the function it enables is pretty novel. This is entirely debatable, and probably matters less than showing were your cursor is in a google doc.

Regardless, I don't think it matters since the author is not in the EU.

Re: What every software developer must know about Unicode in 2023

#514

Post by dwringer »

jfultz wrote: post #172
pif wrote: post #152

Earlier quoted context omitted.

> An extended grapheme cluster is largely defined on "this visually displays as a single unit", which isn't necessarily correct for things like "display size in a monospace font" or "thing that gets deleted when you hit backspace." I'm sorry, but I fail to see how "This visually displays as a single unit" could ever differ from "Display size in a monospace font" or "Thing that gets deleted when you hit backspace".

A couple of cases I'm aware of... * Coding ligatures often display as a single glyph (maybe occupying a single-width character space, or maybe spread out over multiple spaces), but are composed of multiple glyphs. The ligature may "look" like a single character for purposes of selection and cursoring, but it can act like multiple characters when subject to backspacing. * Similarly, I've seen keyboard interfaces for v…

I've always felt ligatures that condense two or more glyphs into something that takes up the space of only one in a monospace font are going beyond what a font should handle and into the realm of what an editor should do. I have several such visual substitutions set up in my .emacs but I don't use fonts that do them on their own.

Re: What every software developer must know about Unicode in 2023

#515

Post by Etherlord87 »

extraduder_ire wrote: post #489
Etherlord87 wrote: post #459

Earlier quoted context omitted.

You're absolutely correct! `length` is ambiguous - you shouldn't have a `time` argument in a `sleep` function; you should have `milliseconds` and/or `seconds` etc.

You could have a Duration argument though. The parallels of string length with the phrase "How long is a piece of string?"[0] make this apparent/amusing. I'm sure I'm not the first person to think that. [0]: https://en.wiktionary.org/wiki/how_long_is_a_piece_of_string

The `Duration` being a type that implements an interface removing the ambiguity? Like a `DateTime` object does? I think it might be useful to have a function returning a collection of information about text, how many unicode points, how many grapheme clusters, how many syllables, vowels, consonants, special characters… But for performance reasons you probably want separate functions that give you just one of these.

Re: What every software developer must know about Unicode in 2023

#517

Post by mostlylurks »

Findecanor wrote: post #350
mostlylurks wrote: post #294

Earlier quoted context omitted.

Do the danes not have the mechanism that is found on Finnish keyboard layouts, where pressing AltGr+Ö yields Ø and AltGr+Ä yields Æ, except in reverse?

Those mappings are not universal. They are present under Linux but not on MS-Windows. I don't know about Mac, but the layout has in the past been slightly different there from Windows also.

Interesting, it's been like a decade since I last used windows, but I had to go and check what layouts are available, since I remember having these combinations on my layout. Apparently those combinations are provided by windows in the "Finnish and Sami" layout, which provides a number of extra letters (not just ones used by the Sami languages) through AltGr+letter combinations. I must have selected that as my layout at some point while I was still using windows, possibly for the purpose of getting easier access to letters like ÆØÕ, and just forgotten it after some time.

Re: What every software developer must know about Unicode in 2023

#518

Post by rurban »

nextaccountic wrote: post #24
rurban wrote: post #17

The Why is "Å" !== "Å" !== "Å"? section still strikes me as wrong. The strings are equal even when the representations differ.

They are logically equal (that is, they represent the same text in an abstract way), but computing this equality in practice is expensive, because you first need to normalize the strings then compare. Most languages, when comparing strings, skip the normalization and just compare string bytes as is (or, if the string is interned, compare just the pointer)

You can easily do the comparison dynamically with checking for combining marks, and then do the proper lookup. No need to normalize everything, or even store the normalized variant. Though in a filesystem or username lookup you would only store it normalized.

Re: What every software developer must know about Unicode in 2023

#519

Post by mike_hearn »

kalleboo wrote: post #485
haberman wrote: post #364

Earlier quoted context omitted.

In that case, it sounds like `length` on Unicode strings simply shouldn't exist, since there is no obvious right answer for it. Instead there should be `codepointCount`, `graphemeCount`, etc.

There are basically 2 places where programmers mostly want the "length" of a string: 1. To save storage space or avoid pathological input, they want to limit the "length" of text input fields. E.g., not allow a name to be 4 KB long 2. To fit something on screen Developers mostly used to western languages can approximate both with "number of letters", but the correct answers are For 1. Limit to bytes to avoid people b…

Presizing buffers, initializing for loop counts ...

Re: What every software developer must know about Unicode in 2023

#520

Post by kalleboo »

mike_hearn wrote: post #519
kalleboo wrote: post #485

Earlier quoted context omitted.

There are basically 2 places where programmers mostly want the "length" of a string: 1. To save storage space or avoid pathological input, they want to limit the "length" of text input fields. E.g., not allow a name to be 4 KB long 2. To fit something on screen Developers mostly used to western languages can approximate both with "number of letters", but the correct answers are For 1. Limit to bytes to avoid people b…

Presizing buffers, initializing for loop counts ...

Buffers should just be working on raw byte arrays without even considering the content (if it's a string or data or whatever).

"for loop counts" depends on what you're doing in the loop...

Post reply on HN