Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A popular but wrong way to convert a string to uppercase or lowercase

devblogs.microsoft.com

Post reply on HN
Print view View original

171–180 of 272 posts

Re: A popular but wrong way to convert a string to uppercase or lowercase

#171

Post by PaulDavisThe1st »

vardump wrote: post #2

As always, Raymond is right. (And as usually, I could guess it's him before even clicking the link.) That said, 99% time when doing upper- or lowercase operation you're interested just in the 7-bit ASCII range of characters. For the remaining 1%, there's ICU library. Just like Raymond Chen mentioned.

He may be right, but approximately 75% of the problems he describes are all Microsoft-ecosystem specific.

In Unix-land we don't use wchar_t or UTF-16, and his article is a good demonstrations of why not.

Re: A popular but wrong way to convert a string to uppercase or lowercase

#172

Post by dennis_jeeves2 »

blenderob wrote: post #3

It is issues like this due to which I gave up on C++. There are so many ways to do something and every way is freaking wrong! An acceptable solution is given at the end of the article: > If you use the International Components for Unicode (ICU) library, you can use u_strToUpper and u_strToLower. Makes you wonder why this isn't part of the C++ standard library itself. Every revision of the C++ standard brings with its…

> There are so many ways to do something and every way is freaking wrong!

Stroustrup, laugheth!

Re: A popular but wrong way to convert a string to uppercase or lowercase

#173

Post by oguz-ismail »

PaulDavisThe1st wrote: post #170
oguz-ismail wrote: post #158

Earlier quoted context omitted.

> 99% of Unicode stuff Does that include context-dependent conversion rules like o'reilly -> O'Reilly?

that is neither up-casing nor-downcasing, but (de)capitalization, which is a significantly more complex task (which ultimately requires up- or down-casing, but a whole lot more before then).

So it doesn't. If Unicode doesn't cover non-trivial forms of case-folding, 99% of Unicode doesn't mean anything.

Re: A popular but wrong way to convert a string to uppercase or lowercase

#174

Post by NoInkling »

mkayokay wrote: post #26
cyxxon wrote: post #17

Small nitpick: the example "LATIN SMALL LETTER SHARP S (“ß” U+00DF) uppercases to the two-character sequence “SS”:³ Straße ⇒ STRASSE" is slightly wrong, it seems to me, as we now do actually have a uppercase version of that, so it should uppercase to "Latin Capital Letter Sharp S" (U+1E9E). The double-S thing is still widely used, though.

Duden mentions this: "Bei Verwendung von Großbuchstaben steht traditionellerweise SS für ß. In manchen Schriften gibt es aber auch einen entsprechenden Großbuchstaben; seine Verwendung ist fakultativ ‹§ 25 E3›." But isn't it also dependent on the available glyphs in the font used? So f.e. it needs to be ensured that U+1E9E exists?

According to Wikipedia:

> "Since 2024 the capital ⟨ẞ⟩ is preferred over ⟨SS⟩."

https://en.wikipedia.org/wiki/%C3%9F

Check reference #5 and compare it to the older wording in reference #12.

Re: A popular but wrong way to convert a string to uppercase or lowercase

#175

Post by yas_hmaheshwari »

vardump wrote: post #2

As always, Raymond is right. (And as usually, I could guess it's him before even clicking the link.) That said, 99% time when doing upper- or lowercase operation you're interested just in the 7-bit ASCII range of characters. For the remaining 1%, there's ICU library. Just like Raymond Chen mentioned.

Wow, I came here to write exactly that, and its heartening to see that I am not crazy

Just reading the title, with microsoft.com in bracket, I knew two things: 1. It would be written by Raymond Chen 2. That article is going to be awesome

Re: A popular but wrong way to convert a string to uppercase or lowercase

#176

Post by Towaway69 »

tedunangst wrote: post #95
zahlman wrote: post #82

Earlier quoted context omitted.

Python's strings have uppercase, lowercase and case-folding methods that don't choke on this. They don't use UTF-16 internally (they can use UCS-2 for strings whose code points will fit in that range; while a string might store code points from the surrogate-pair range, they're never interpreted as surrogate pairs, but instead as an error encoding so that e.g. invalid UTF-8 can be round-tripped) so they're never worr…

But that's wrong. The upper case for ß is ẞ.

Isn't uppercase for ß just ß - i.e. it's its own uppercase character?

Re: A popular but wrong way to convert a string to uppercase or lowercase

#177

Post by lmm »

NBJack wrote: post #165
lmm wrote: post #150

Earlier quoted context omitted.

> What if your game needs to talk to a server and do some string manipulation in between requests? What conceivable reason would there be to ever need to do that? If the server takes commands in upper case, then have them in upper case from the start. If the server takes commands in lower case, have them in lower case from the start. If the server specifies that you need to invert the case of its response to use in t…

Word censoring? Ease of use? Console commands (i.e. from Quake to minecraft)?

> Word censoring?

Should only ever be needed for text from the user, and in that case, as GP said, find a way to examine it as-is, don't "convert".

> Ease of use?

What ease of use? When has futzing around with case ever made anything easier?

> Console commands (i.e. from Quake to minecraft)?

Why would those necessitate changing case?

Re: A popular but wrong way to convert a string to uppercase or lowercase

#178

Post by alkonaut »

Handle text in two ways: either it's controlled by you and you can do simple, efficient, and naive processing, or it's not (it's translated resources, or user input) and you can't.

For the former case, you don't need any complex logic. A very typical example would be: i'm serializing a field or constructing a url so I want the variable name "Someproperty" as a lower case string. The lowercase transform is completely naive. I know exactly what the range of possible characters are and they aren't going to be Turkish or emoji, not least because I have asserted they won't be. And THIS is what the regular programming functions for upper/lower case are for. They are important, and they are most often correct. Because for all the other cases (i18n, user input, ...) you probably don't want to do toUpper/toLower at all to begin with!

Example, if you present a message to the user from resources so your code is translate("USER_DIALOG_QUESTION_ABOUT_FISH") which you want to lookup knowing it will be in sentence case, and present as uppercase, what will you do? Here you likely can't, and shouldn't, do toUpper(translate(resourceKey)). Just use two resources if you want to correctly transform text. The toUpper function isn't made for this.

Trying to use a complex i18n-ready toUpper/toLower only helps part of the way. It still might not understand whether two S are contracted or whether something is a proper Noun and must stay capitalized. So it adds complexity and still isn't correct. Just use two resources!

Re: A popular but wrong way to convert a string to uppercase or lowercase

#179

Post by HPsquared »

klyrs wrote: post #161
HPsquared wrote: post #85

I thought this was going to be about adding or subtracting 32. Old school.

I do hope you mean bitwise "addition" and "subtraction" -- (c => c&0xdf) or (c => c|0x20)

Tbh I come at this as a plebeian Excel user

Re: A popular but wrong way to convert a string to uppercase or lowercase

#180

Post by Smaug123 »

cheema33 wrote: post #169
SleepyMyroslav wrote: post #73

In gamedev there is simple rule: don't try to do any of that. If it is text game needs to show to user then every version of the text that is needed is a translated text. Programmer will never know if context or locale will need word order changes or anything complicated. Just trust the translation team. If text is coming from user - then change design until its not needed to 'convert'. There are major issues just to…

> In gamedev there is simple rule: don't try to do any of that. I am not in gamedev, but I frequently have to develop middleware that takes in user entered data and formats it in a way that will import into a 3rd party system without errors. And that sometimes means changing the case on strings. In my experience as a developer, this is very very common requirement. Luckily I am not forced to use a low level language…

If you're putting data into a third-party system, you might want `ToUpperInvariant`, not `ToUpper`. (Just checking that you know the difference, because most people don't!)
Post reply on HN