It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

Post by arcticbull » Fri, Aug 22, 2025, 9:04 AM UTC

setr wrote: post #58

arcticbull wrote: post #50

Earlier quoted context omitted.

Substring operations (and more generally the universe of operations where there is more than one string involved) are a whole other kettle of fish. Unicode, being a byte code format more than what you think of as a logical 'string' format, has multiple ways of representing the same strings. If you take a substring of a(bc) and compare it to string (bc) are you looking for bitwise equivalence or logical equivalence? I…

I’m fairly positive the answer is trivially logical equivalence for pretty much any substring operation. I can’t imagine bitwise equivalence to ever be the “normal” use case, except to the implementer looking at it as a simpler/faster operation I feel like if you’re looking for bitwise equivalence or similar, you should have to cast to some kind of byte array and access the corresponding operations accordingly

Post by account42 » Fri, Aug 22, 2025, 9:07 AM UTC

arcticbull wrote: post #35

DavidPiper wrote: post #11

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted: - Number of bytes this will be stored as in the DB - Number of monospaced font character blocks this string will take up on the screen - Number of bytes that are actually being stored in memory "String length" is just a…

Taking this one step further -- there's no such thing as the context-free length of a string. Strings should be thought of more like opaque blobs, and you should derive their length exclusively in the context in which you intend to use it. It's an API anti-pattern to have a context-free length property associated with a string because it implies something about the receiver that just isn't true for all relevant usage…

Post by account42 » Fri, Aug 22, 2025, 9:10 AM UTC

baq wrote: post #17

DavidPiper wrote: post #11

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted: - Number of bytes this will be stored as in the DB - Number of monospaced font character blocks this string will take up on the screen - Number of bytes that are actually being stored in memory "String length" is just a…

ASCII is very convenient when it fits in the solution space (it’d better be, it was designed for a reason), but in the global international connected computing world it doesn’t fit at all. The problem is all the tutorials, especially low level ones, assume ASCII so 1) you can print something to the console and 2) to avoid mentioning that strings are hard so folks don’t get discouraged. Notably Rust did the correct th…

Post by CorrectHorseBat » Fri, Aug 22, 2025, 9:13 AM UTC

guappa wrote: post #45

taneq wrote: post #26

Earlier quoted context omitted.

If you're playing at this level, you need to define: - letter - word - 5 :P

Eh in macedonian they have some letters that in russian are just 2 separate letters

Post by amake » Fri, Aug 22, 2025, 9:14 AM UTC

degamad wrote: post #52

eru wrote: post #28

Earlier quoted context omitted.

You could use a standard that always uses eg 4 bytes per character, that is much easier to parse than UTF-8. UTF-8 is so complicated, because it wants to be backwards compatible with ASCII.

It's not just the variable byte length that causes an issue, in some ways that's the easiest part of the problem. You also have to deal with code points that modify other code points, rather than being characters themselves. That's a huge part of the problem.