Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

hsivonen.fi

Post reply on HN
Print view View original

11–20 of 287 posts

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#11

Post by DavidPiper »

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted:

- Number of bytes this will be stored as in the DB

- Number of monospaced font character blocks this string will take up on the screen

- Number of bytes that are actually being stored in memory

"String length" is just a proxy for something else, and whenever I'm thinking shallowly enough to want it (small scripts, mostly-ASCII, mostly-English, mostly-obvious failure modes, etc) I like grapheme cluster being the sensible default thing that people probably expect, on average.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#12

Post by ale42 »

cmeacham98 wrote: post #5
bstsb wrote: post #2

ironic that unicode is stripped out the post's title here, making it very much wrong ;) for context, the actual post features an emoji with multiple unicode codepoints in between the quotes

Funny enough I clicked on the post wondering how it could possibly be that a single space was length 7.

Maybe it isn't a space, but a list of invisible Unicode chars...

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#14

Post by yread »

ale42 wrote: post #12
cmeacham98 wrote: post #5

Earlier quoted context omitted.

Funny enough I clicked on the post wondering how it could possibly be that a single space was length 7.

Maybe it isn't a space, but a list of invisible Unicode chars...

It could also be a byte length of a 3 byte UTF-8 BOM and then some stupid space character like f09d85b3

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#15

Post by kazinator »

Why would I want this to be 17, if I'm representing strings as array of code points, rather than UTF-8?

TXR Lisp:

  1> (len " ")
  5
  2> (coded-length " ")
  17
(Trust me when I say that the emoji was there when I edited the comment.)

The second value takes work; we have to go through the code points and add up their UTF-8 lengths. The coded length is not cached.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#16

Post by eastbound »

cmeacham98 wrote: post #5
bstsb wrote: post #2

ironic that unicode is stripped out the post's title here, making it very much wrong ;) for context, the actual post features an emoji with multiple unicode codepoints in between the quotes

Funny enough I clicked on the post wondering how it could possibly be that a single space was length 7.

It can be many Zero-Width Space, or a few Hair-Width Space.

You never know, when you don’t know CSS and try to align your pixels with spaces. Some programers should start a trend where 1 tab = 3 hairline-width spaces (smaller than 1 char width).

Next up: The tag.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#17

Post by baq »

DavidPiper wrote: post #11

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted: - Number of bytes this will be stored as in the DB - Number of monospaced font character blocks this string will take up on the screen - Number of bytes that are actually being stored in memory "String length" is just a…

ASCII is very convenient when it fits in the solution space (it’d better be, it was designed for a reason), but in the global international connected computing world it doesn’t fit at all. The problem is all the tutorials, especially low level ones, assume ASCII so 1) you can print something to the console and 2) to avoid mentioning that strings are hard so folks don’t get discouraged.

Notably Rust did the correct thing by defining multiple slightly incompatible string types for different purposes in the standard library and regularly gets flak for it.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#18

Post by sigmoid10 »

DavidPiper wrote: post #11

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted: - Number of bytes this will be stored as in the DB - Number of monospaced font character blocks this string will take up on the screen - Number of bytes that are actually being stored in memory "String length" is just a…

I have wanted string length many times in production systems for language processing. And it is perfectly fine as long as whatever you are using is consistent. I rarely care how many bytes an emoji actually is unless I'm worried about extreme efficiency in storage or how many monospace characters it uses unless I do very specific UI things. This blog is more of a cautionary tale what can happen if you unconsciously mix standards e.g. by using one in the backend and another in the frontend. But this is not a problem of string lengths per se, they are just one instance where modern implementations are all over the place.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#20

Post by xg15 »

DavidPiper wrote: post #11

I think that string length is one of those things that people (including me) don't realise they never actually want. In a production system, I have never actually wanted string length. I have wanted: - Number of bytes this will be stored as in the DB - Number of monospaced font character blocks this string will take up on the screen - Number of bytes that are actually being stored in memory "String length" is just a…

It gets more complicated if you do substring operations.

If I do s.charAt(x) or s.codePointAt(x) or s.substring(x, y), I'd like to know which values for x and y are valid and which aren't.

Post reply on HN