It’s not wrong that "🤦🏼‍♂️".length == 7 (2019)

Post by cylemons » Fri, Jun 02, 2023, 12:04 PM UTC

chrismorgan wrote: post #127

xeeeeeeeeeeenu wrote: post #34

I'm not a fan of "everything you know about X is wrong" articles. Very often they try to present some little tidbit of knowledge as a revelation and mislead the reader in the process. In this case, the tidbit is: "grapheme clusters exist and they are useful". The misleading part is that the article draws a false equivalence between what the author calls "UTF-32 code units" and UTF-16 code units. UTF-32 code units are…

> UTF-16 code units, on the other hand, are an implementation detail of UTF-16. Would that it were only so. Instead, UTF-16 ruined Unicode for everyone with the abomination that is surrogates , and almost nothing that deals with UTF-16 actually asserts well-formedness, and ill-formed UTF-16 cannot be represented in UTF-8 or UTF-32. UTF-32 and UTF-8 code units are truly implementation details of their encodings, as ot…

Post by dpiers » Fri, Jun 02, 2023, 12:11 PM UTC

tylergetsay wrote: post #190

Interesting; emojis also make SMS really weird. according to twilio including a emoji will change the character limit from 160 to 70

Post by eviks » Fri, Jun 02, 2023, 12:20 PM UTC

pavlov wrote: post #179

eviks wrote: post #174

Earlier quoted context omitted.

Showing unintuitive results in a small number of cases always beats showing unintuitive results for all the cases. The Swift approach can also asymptotically reach perfection. The other approach will be broken forever. These are categorical improvements in design

The Swift approach can't reach perfection in isolation because data from the future can always break it. That's why in the article you see Swift running on Ubuntu 14.04 returning len==2 while the same code on Ubuntu 18.04 returns len==1 for the same emoji string. IMO that's a big philosophical question here: do we accept that "string length" means something you can't compute for arbitrary strings unless your code is…

Post by hardware2win » Fri, Jun 02, 2023, 12:28 PM UTC

alkonaut wrote: post #134

Really the correct way to design string APIs would be to not have an ambiguous "length" at all, but to always require specifying whether you want UTF8-bytes, memory bytes, code points, graphemes, whatever. However such an API would be pretty cumbersome because for all non-edge cases (read: a western language and a reasonable encoding that language - which when looking at world demographics is a very narrow way of say…

Post by wtetzner » Fri, Jun 02, 2023, 12:43 PM UTC

zarzavat wrote: post #155

arcticbull wrote: post #24

Earlier quoted context omitted.

It is wrong that "{emoji}".length == 7 -- but it's wrong because there's no such thing as the 'length' of a string out of context. A string should be viewed as an opaque data type with views into it depending on what you're trying to do. You can have its length in the context of storage/retrieval/transmission (UTF-8 byte count), its length in the context of parsing (code points), its length in the context of editing…

There are three notions of length that make sense: 1. UTF-8 byte length 2. Code point count 3. Extended grapheme cluster count #3 makes sense for users but it doesn’t make sense for programs which often need to work at the code point level. I expect programming language string length to obey the law: len(a ++ b) = len(a) + len(b) For example, if I concatenate a two strings, one containing an “e” and one containing a…