Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why can't you reverse a string with a flag emoji?

davidamos.dev

Post reply on HN
Print view View original

221–230 of 247 posts

Re: Why can't you reverse a string with a flag emoji?

#221

Post by account42 »

emodendroket wrote: post #181
shalmanese wrote: post #163

Earlier quoted context omitted.

It doesn't make sense but there's also no way to fix it now. Once the Han characters were unified, there's no non-trivial way to ununify them.

To an extent that's true, but introducing national variant characters in addition to the unified ones would at least allow careful writers to avoid the problem.

Exactly, this is not rocket science: Introduce variantes of the affected characters in unicode (either variant selectors or distinct codepoints, doesn't matter too much but variant selectors could allow falling back to the old context-based detection). Then wait for software to be updated to use the variants based on the input language. This allows the writer to verify the variant used which will then be the same in all contexts.

Re: Why can't you reverse a string with a flag emoji?

#222

Post by account42 »

avianlyric wrote: post #124
wahern wrote: post #50

Earlier quoted context omitted.

There's a specification problem here. I like to say that a "string" isn't a data structure, it's the absence of one. Discussing "strings" is pointless. It follows that comparing programming languages by their "string" handling is likewise pointless. Case in point: a "struct" in languages like C and Rust is literally a specification of how to treat segments of a "string" of contiguous bytes.

In languages like C “string” isn’t a proper data structure, it’s a `char` array, which itself is little more than a `int` array or `byte` array. But these languages don’t provide true “string” support. They just have a vaguely useful type alias that renames a byte array to a char array, and a bunch of byte array functions that have been renamed to sound like string functions. In reality all the language supports are…

Having your strings be conceptually made up of UTF-8 code units makes them no less strings than those made up of Unicode code points. As this article shows, working with code points is often not the right abstraction anyway and you need to up all the way to grapheme clusters to have anything close to what someone would intuitively call a character. Calling a code point a character is not more correct or useful than calling a code unit a char.

All you gain by having Unicode code point strings is the illusion of Unicode support until you test anything that uses combining characters or variant selectors. In essence, languages opting for such strings are making the same mistake at Windows/Java/etc. did when adopting UTF-16.

Re: Why can't you reverse a string with a flag emoji?

#223

Post by account42 »

samatman wrote: post #136
wahern wrote: post #50

Earlier quoted context omitted.

There's a specification problem here. I like to say that a "string" isn't a data structure, it's the absence of one. Discussing "strings" is pointless. It follows that comparing programming languages by their "string" handling is likewise pointless. Case in point: a "struct" in languages like C and Rust is literally a specification of how to treat segments of a "string" of contiguous bytes.

We would all be better off if this were actually true. Tragically, in C, a string is just barely a data structure, because it must have \0 at the end. If it were the complete absence of a data structure, we would need some way to get at the length of it, and could treat a slice of it as the same sort of thing as the thing itself.

You can work with pointer+length or begin+end pairs in C just fine - it's just annoying. But you can always upgrade to C++ and use std::string_view to abstract that for you if you want.

Re: Why can't you reverse a string with a flag emoji?

#224

Post by mFixman »

kingcharles wrote: post #147
ghostly_s wrote: post #105

Earlier quoted context omitted.

This was the only part that was surprising to me, and as it turns out my surprise mostly stems from still not really understanding how the United Kingdom works.

I was born and raised in the UK and I still can't explain to people how it works. Are England, Scotland, Wales and Northern Ireland separate countries? Is the UK a country? It's countries all the way down.

The UK is an unitary state with 3 autonomous areas that have different devolved levels of self-governance, similar to Italy or (in theory) China.

The difficult part comes from calling these autonomous areas "countries".

Re: Why can't you reverse a string with a flag emoji?

#225

Post by account42 »

viktorcode wrote: post #52
jerf wrote: post #6

So, in terms of acing interviews, increasingly one of the best answers to the question "Write some code that reverses a string" is that in a world of unicode, "reversing a string" is no longer possible or meaningful. You'll probably be told "oh, assume US ASCII" or something, but in the meantime, if you can back that up when they dig into it, you'll look really smart.

You certainly can. `print(String(flag.reversed()))` in Swift reverses emojis correctly.

How does it handle the ASCII examples in https://news.ycombinator.com/item?id=30108184

And more importantly: What is the use case for a reversed string?

Re: Why can't you reverse a string with a flag emoji?

#226

Post by account42 »

lloeki wrote: post #26
Spivak wrote: post #11

Earlier quoted context omitted.

Reversing a string is still meaningful. Take a step back outside the implementation and imagine handing a Unicode string to a human. They could without any knowledge look at the characters they see and produce the correct string reversal. There is a solution to this which is to compute the list of grapheme clusters, and reverse that. https://unicode.org/reports/tr29/

Should it reverse a BOM as well or keep it first?

Remove it since the BOM is a hack to deal with shitty transfer encodings (i.e. UTF-16LE vs. UTF-16BE) and useless for UTF-8.

Re: Why can't you reverse a string with a flag emoji?

#227

Post by Vindicis »

coreyp_1 wrote: post #53

If you think the Unicode flag emoji take a lot of bytes, then consider the family emoji! ( https://unicode.org/emoji/charts/full-emoji-list.html#family ) I'm in the process of designing a scripting language and implementing it in C++. I plan to put together a YouTube series about it. (Doesn't everyone want to see Bison and Flex mixed with proper unit tests and C++20 code?) Due to my future intended use case, I needed…

I know how that feels. I wrote a little c++ program to fetch data in Unicode from a dB and then normalize it to ascii to be used for analytic purposes. A lot faster to do it on ascii than trying to handle all the fun cases of how many ways can an e etc... be input. ICU to the rescue! Took a couple weeks of getting up to speed as ICU itself wasn't too bad to figure out. But, you find out very quickly that to use it, you need to have a good understanding of a number of the Unicode technical reports to actually understand how to make use of it. Fun times indeed.

Re: Why can't you reverse a string with a flag emoji?

#228

Post by account42 »

masklinn wrote: post #90
otagekki wrote: post #57

If flag emojis are really a combination of 2 special characters, the reversal of the U.S. flag should result in having the Soviet Union flag.

> the reversal of the U.S. flag should result in having the Soviet Union flag. Except it has been deleted from the ISO 3166-2 registry, so not having it is perfectly valid (arguably more so than having it).

No, that only shows that the ISO 3166-2 registry is a bad basis for Unicode flags since having things lose meaning over time should not be acceptable for a text encoding.

Flags have another issue here in that they can change even when the country stays the same - a recent example here being Afghanistan, but also France who recently changed the official shades of the colors in their flag. Ideally you'd want a new Unicode representation for any changed flags in order to not retroactively change the meaning in old documents.

Re: Why can't you reverse a string with a flag emoji?

#229

Post by account42 »

progbits wrote: post #60

Semi-related (about length of emoji "characters", not reversing): https://hsivonen.fi/string-length/ Previously discussed: https://news.ycombinator.com/item?id=20914184 https://news.ycombinator.com/item?id=26591373 As for this article & Python - as usual it is biasing towards convenience and implicit behavior rather than properly handling all edge cases. Compare with Rust where you can't "reverse" a string - that is…

> Sadly the grapheme segmentation is not part of standard library, at least yet

Seeing as grapheme segmentation is a moving target that only makes sense.

Re: Why can't you reverse a string with a flag emoji?

#230

Post by account42 »

cmyr wrote: post #146

Something I haven't seen mentioned yet is one of the most annoying things about regional indicator symbols, which is that interpreting them correctly requires arbitrary backtracking, and handling this correctly is very annoying for things like text fields. Basically: A single, unpaired RIS counts as a single grapheme. Similarly, a pair of RIS count as a single grapheme. Now imagine if your cursor position is after an…

> Isn't hindsight a wonderful thing?

Gladly, the creators of UTF-18 did have that foresight so at least we don't have this problem at the code unit -> code point level.

Post reply on HN