71–80 of 272 posts
Earlier quoted context omitted.
Different parts of a string can be in different languages too[1]. The lowercase of "DON'T FUSS ABOUT FUSSBALL" is "don't fuss about fußball". Unless you're in Switzerland. [1] https://en.wikipedia.org/wiki/Code-switching
I thought the German language deprecated the use of ß years ago, no? I learned German for a year and that's what the teacher told us, but maybe it's not the whole story
The real insights here are that strings in C++ suck and UTF-16 is extremely unintuitive.
As for UTF-16, well, I don't know that UTF-8 is a whole lot more intuitive:
> And for UTF-8 data, you have the same issues discussed before: Multibyte characters will not be converted properly, and it breaks for case mappings that alter string lengths.
If it is text game needs to show to user then every version of the text that is needed is a translated text. Programmer will never know if context or locale will need word order changes or anything complicated. Just trust the translation team.
If text is coming from user - then change design until its not needed to 'convert'. There are major issues just to show user back what he entered! Because the font for editing and displayed text could be different. Not even mentioning RTL and other issues.
Once ppl learn about localization the questions like why a programming language does not do this 'simple text operation' are just a newcomer detector. :)
Earlier quoted context omitted.
That explains why there are two functions, one for ascii and one for unicode. That doesn't explain why the unicode functions are hard to use (per the article).
Because human language is hard to boil down to a simple computing model and the problem is underdefined, based on naive assumptions. Or perhaps I should say naïve.
Earlier quoted context omitted.
Because human language is hard to boil down to a simple computing model and the problem is underdefined, based on naive assumptions. Or perhaps I should say naïve.
Well pretty much every other more recent language solved that problem.
Earlier quoted context omitted.
Here's an example. Hypothetically say you want to build an HTML parser. You might encounter tags like , , , etc., but you want to perform a hash table lookup. So first you're going to normalize to either lower- or uppercase.
Ah, i see, we disagree on what is "human language". An abbreviation like HTML and it's different capitalisations to me sound a lot like a feature of human language.
The real insights here are that strings in C++ suck and UTF-16 is extremely unintuitive.
Strings in C++ standard library do suck (and C++ is my favorite language). As for UTF-16, well, I don't know that UTF-8 is a whole lot more intuitive: > And for UTF-8 data, you have the same issues discussed before: Multibyte characters will not be converted properly, and it breaks for case mappings that alter string lengths.
It is issues like this due to which I gave up on C++. There are so many ways to do something and every way is freaking wrong! An acceptable solution is given at the end of the article: > If you use the International Components for Unicode (ICU) library, you can use u_strToUpper and u_strToLower. Makes you wonder why this isn't part of the C++ standard library itself. Every revision of the C++ standard brings with its…
I don't think it's a C++ problem. You just can't transform anything developed in "ancient" times to unicode aware in a single swoop. On the other hand, libicu is 37MB by itself, so it's not something someone can write in a weekend and ship. Any tool which is old enough will have a thousand ways to do something. This is the inevitability of software and programming languages. In the domain of C++, which has a size mam…
Isn't that mostly just from tables derived from the Unicode standard?
It is issues like this due to which I gave up on C++. There are so many ways to do something and every way is freaking wrong! An acceptable solution is given at the end of the article: > If you use the International Components for Unicode (ICU) library, you can use u_strToUpper and u_strToLower. Makes you wonder why this isn't part of the C++ standard library itself. Every revision of the C++ standard brings with its…
I don't think it's a C++ problem. You just can't transform anything developed in "ancient" times to unicode aware in a single swoop. On the other hand, libicu is 37MB by itself, so it's not something someone can write in a weekend and ship. Any tool which is old enough will have a thousand ways to do something. This is the inevitability of software and programming languages. In the domain of C++, which has a size mam…
Even for Python it took well over a decade, and people still complain about the fact that they don't get to treat byte-sequences transparently as text any more - as if they want to wrestle with the `basestring` supertype, getting `UnicodeDecodeError` from an encoding operation or vice-versa, trying to guess the encoding of someone else's data instead of expecting it to be decoded on the other side....
But in C++ (and in C), you have the additional problem that the 8-bit integer type was named for the concept of a character of text, even though it clearly cannot actually represent any such thing. (Not to mention the whole bit about `char` being a separate type from both `signed char` and `unsigned char`, without defined signedness.)
Earlier quoted context omitted.
Why do you need upper- or lowercase conversion in cases that have nothing to do with human language?
Here's an example. Hypothetically say you want to build an HTML parser. You might encounter tags like , , , etc., but you want to perform a hash table lookup. So first you're going to normalize to either lower- or uppercase.