Earlier quoted context omitted.
windows probably defaults to latin-1
the default windows encoding is UTF-16, a long time ago it was Windows-1252 https://en.wikipedia.org/wiki/Windows-1252
or CP-1251, in some locations.
I couldn't debug the code because of my name
71–80 of 300 posts
Re: I couldn't debug the code because of my name
#72Earlier quoted context omitted.
This is the modern, post-ASCII computing world, we should no longer be willing to settle for the lowest-common-denominator of ASCII-only strings. There's no excuse for actively supported, paid products to have these problems today.
Especially if those products are developed by a company from Russia, where Cyrillic is used. For me, a Russian myself, this situation is honestly ridiculous.
Do you write "if" statements in Cyrillic when you write in ?
Re: I couldn't debug the code because of my name
#73Using non-ascii characters in file paths, toolchain config files, and other non-display contexts is just asking for trouble, even if it is your name...
Unfortunately, it's true, most toolchains are stuck in the past, and don't deal with non-ascii characters or even spaces very well. In fact, I just learned that spaces in .deskop files values could cause trouble after a long debugging.
But it's a shame.
In Europe, we do have a lot of non-ascii characters everywhere. Ubuntu puts a "Vidéo" and a "Téléchargements" directory in my $HOME because I'm french. If I were to use my name as my username I would have even more troubles.
I'm careful with not using special chars in names for work, but it feels like I'm a girl trying to not dress sexy in the wrong part of town: necessary, but I shouldn't have to do this, and it's definitely the others to blame.
All in all, I thank the Gods of encoding for Python 3 unicode handling. Having a scripting language that does the right thing out of the box is wonderful on this side of the pond.
Re: I couldn't debug the code because of my name
#74It's somewhat common to see videogames issue a patch shortly after release where they fix crashes due to non-ASCII Windows usernames or non-English locales. I'm not sure what the root cause of the confusion is, other than text strings being hard in general.
It's easy to think the answer is "just UTF-8 everything" but unfortunately the long and twisty history of filesystems means that's not the correct answer, and the "correct answer" is really hard to write down quickly. If you never display the filename, the answer is to treat existing filenames as bags of bytes, but that breaks down as soon as you need to display them, or if you need to manipulate them by appending un…
Rust uses WTF-8 on Windows for OsStr[ing] and Path[Buf]. It's zero-overhead to cast from &str to &OsStr/&Path to &[u8] (though converting WTF-8 to UTF-16 costs an extra operation when performing a Win32 function call). However this doesn't solve the inability to round-trip "possibly-valid UTF-8/16" to "Unicode text" and back (though Python's surrogateescape might be one viable approach).
Other libraries handle this even worse than Rust. On Linux (filenames are bytes), Qt is unable to open files with invalid UTF-8 names, while GTK can open them (but shows an "invalid encoding" message instead of the original filename), which I think is a good-enough approach.
Re: I couldn't debug the code because of my name
#75Re: I couldn't debug the code because of my name
#76For a list of strings that often cause problems to, e.g., add to a test suite, see https://github.com/minimaxir/big-list-of-naughty-strings
Very handy. My previous simple test-case was simply a selection from this well-known text-file which is simply a collection of somewhat uncommon unicode characters, usually used for rendering tests.
https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt
But this set of strings is specifically designed to cause edge-case errors.
Also don't forget Spolsky's seminal "The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)".
https://www.joelonsoftware.com/2003/10/08/the-absolute-minim...
Re: I couldn't debug the code because of my name
#77Earlier quoted context omitted.
"You're holding it wrong" The problem is the technology, not the user using it in a reasonable way. ł is older than computers and the only reason computers struggle with it is lack of foresight or choosing to make things harder for most of the world by some of the people involved early on.
Obviously the IDE is at fault here. Rider has a bug with Unicode. BUT, there is an easy workaround to avoid all Unicode related bugs: don't use Unicode. If that's morally objectionable for you, then you can keep fighting this fight.
I think it's reasonable to find that morally objectionable: English is the only language* that can be fully represented in ASCII, so pretending that ASCII is all you need excludes a large part of the world.
* yes, by and large. Many languages make do, but even the European languages that use the same script as English cannot be fully represented:
- Pretty much all mainland European languages use accents (simple example, in Spanish el and él are different words)
- French misses ç
- German/Swiss/Austrian misses ß
- Spanish misses ñ
- Dutch misses ij
Re: I couldn't debug the code because of my name
#78The domain name to the website is all ascii..
If you use a Microsoft account to set up windows then you have no control over the local username.
This is exactly why I don't do that initially - I don't mind my account being linked - but I've been bitten by the home path bugs multiple times, I unplug my pc during setup
Re: I couldn't debug the code because of my name
#79Isn't it bizarre that we have self-driving cars, the ISS, and phones with 50 megapixel cameras but still struggle with character encoding?
Character encoding is in a special class of problems. Like time handling.
If you pick up a halfway non-ancient framework in a somewhat common language with a somewhat non-terrible persistence like postgres, you just don't have problems. Just don't care, and it just works.
But it's super easy to derail that fragile correctness with something like MySQLs utf8-ish handling, or some OS's path handling, or 'efficiency', or a user or frontend dev submitting data in a wrong encoding. And then it gets mangled. And then the user is unhappy.
At that point, it becomes very hard to argue why one of the two things is wrong, and the other is not. While the user argues the other way around. Because both look correct, if you look from the right angle. And the only reason why I am right is because of some standard, while the customer is right because of money.
And yes, it is very 'surprising' why our software now functions correctly for russian or greek customers.
Re: I couldn't debug the code because of my name
#80Many years ago I could not access the apple developer panel because of the umlaut in my last name. It was eventually fixed but I was quite surprised that such a large company would run into such a basic issue.
My last name has an apostrophe in it which Apple apparently loves to embed directly into their JavaScript unescaped. For a long time neither I nor Apple could look up AppleCare status on my stuff as they were all linked to my Apple ID. The portal would thus require me to login, but then would just show a partially rendered page as my last name was causing an JS syntax error.
Hmm, it sure sounds like John alert(1);Doe (Bobby Tables' distant cousin) should sign up for an Apple account. An XSS attack which could target the AppleCare reps' machines could be catastrophically bad...