I couldn't debug the code because of my name • Hacker News Board

Re: I couldn't debug the code because of my name

#191

Post by makeitdouble » Thu, Oct 21, 2021, 2:53 AM UTC

gumby wrote: post #91

umvi wrote: post #9

Using non-ascii characters in file paths, toolchain config files, and other non-display contexts is just asking for trouble, even if it is your name...

This is blaming the victim

This is reality though.

As much as I wish we lived in a better world where name characters were better handled, using anything outside of [a-ZA-Z]{12} as a username is a world of hurt. Some people just realize it later than others.

So yes, you shouldn't think of your handlename as your name, it's just another identifier, and choosing simple handle names is a life skill at this point.

Re: I couldn't debug the code because of my name

#192

Post by perl4ever » Thu, Oct 21, 2021, 3:08 AM UTC

pezezin wrote: post #186

perl4ever wrote: post #155

Earlier quoted context omitted.

A Polish relative of mine used to just gave an arbitrary substitute name (e.g. "Dave Smith") for restaurant reservations, because even if they could write his last name, they wouldn't be able to pronounce it. My sibling has a name that has an accent, and just enters it with the plain letter most of the time. The name was once rare and "ethnic", but became popular a generation later so people know how to pronounce it…

I understand your troubles, I'm from Spain so I have two family names and my given name has an accent. Now that I'm living in Japan, it's an endless source of fun. Regarding Chinese names and uncommon characters, Japan has the same problem. It's specially problematic for place names, with some kanji used to write the name of a single place in the whole country! I used to write in a place with such an obscure kanji th…

Because nearly everything in the US shoehorns things into three boxes, virtually every place my name is recorded on something important is different.

I could've been consistent in using two or three out of four, but when I was younger I was intimidated by forms that say you must enter your "full legal name", so I would, and they would mangle it unpredictably.

Checking account, credit card, drivers license, and property deed, each one different.

Well, in fact, my social security card and my birth certificate don't match, so I was doomed from the start.

It gives me some sympathy for places that try to regulate names to avoid parents doing something too goofy.

I sometimes wonder if there will come a day when all the databases will stop allowing discrepancies, and it won't matter to the powers that be, because it's such a tiny percentage of the population that becomes "unpersons".

Re: I couldn't debug the code because of my name

#193

Post by Dunedan » Thu, Oct 21, 2021, 3:09 AM UTC

amarshall wrote: post #70

For a list of strings that often cause problems to, e.g., add to a test suite, see https://github.com/minimaxir/big-list-of-naughty-strings

For finding bugs caused by unexpected inputs I also find property based testing very useful. For Python there is the excellent hypothesis library for doing that: https://hypothesis.readthedocs.io/en/latest/

Re: I couldn't debug the code because of my name

#194

Post by needle0 » Thu, Oct 21, 2021, 3:09 AM UTC

xwdv wrote: post #6

What’s wrong with just writing it as Mikolaj? It’s not like it’s a kanji or something.

Then there are the people whose names ARE in Kanji, thankyouverymuch. Ah, no big deal, there's only around 1.6 billion of us.

Re: I couldn't debug the code because of my name

#195

Post by PrivateButts » Thu, Oct 21, 2021, 3:22 AM UTC

Similar to this, Node and NPM get very temperamental when you have a User folder with a space in it. I gave up on the community workarounds and just created a new account and copied my files over to fix it.

Re: I couldn't debug the code because of my name

#196

Post by deathanatos » Thu, Oct 21, 2021, 3:28 AM UTC

nightpool wrote: post #143

geofft wrote: post #138

Earlier quoted context omitted.

Python 3 usually handles this correctly, and I'm a little bit confused what's going on in the article, exactly. For UNIX path names (and other OS data like environment variables), Python uses the "surrogateescape" error handling method, which does exactly what you ask. Any byte sequence can be converted to a string. If it decodes as valid UTF-8, it will do that. If it hits a byte that does not decode as valid UTF-8 (…

The incorrect docker-compose file was generated by Java (Jetbrains) but consumed by Python (docker-compose). The GP comment was complaining about Python's strict Unicode consumption, not Java's invalid Unicode generation.

The Docker compose file is YAML. My reading of YAML's standard is that it must be in one of the Unicode encodings, and the smell I get from the article is that it is probably in windows-1250 (the CP Windows would use for Polish; Mikołaj is a Polish name, 0xb3, the octet in the error, is the Windows-1250 encoding of "ł"); thus, it isn't valid YAML.

I'm not sure what sane behavior Python could have here besides errorring.

> EVERY language should _try_ to handle Unicode such that if a data sequence were valid before it remains valid after.

This sequence was never valid, and never will be.

> in the article's case, the correct answer is GIGO. Just pass it through and hope it continues to work.

Dear God, no; emit a diagnostic and abort. Countless decades of existing code have shown time and again that "plow forward with some hot garbage" is not a good idea. But that ignores that … that that isn't how any of this works; the YAML parse is going to want to emit strings, which the incoming data isn't.

Re: I couldn't debug the code because of my name

#197

Post by [deleted] » Thu, Oct 21, 2021, 3:40 AM UTC

geofft wrote: post #178

nightpool wrote: post #143

Earlier quoted context omitted.

The incorrect docker-compose file was generated by Java (Jetbrains) but consumed by Python (docker-compose). The GP comment was complaining about Python's strict Unicode consumption, not Java's invalid Unicode generation.

Oh, I see. But if it was UTF-8 it would have worked... I guess the problem is that JetBrains is generating the file in (e.g.) Windows-1252, and Python needs to be told that? Does it work if you set the environment variable PYTHONENCODING to cp1252? (I suppose I should either contact the author, or try it myself...)

[deleted]

Re: I couldn't debug the code because of my name

#198

Post by deathanatos » Thu, Oct 21, 2021, 3:45 AM UTC

geofft wrote: post #178

nightpool wrote: post #143

Earlier quoted context omitted.

The incorrect docker-compose file was generated by Java (Jetbrains) but consumed by Python (docker-compose). The GP comment was complaining about Python's strict Unicode consumption, not Java's invalid Unicode generation.

Oh, I see. But if it was UTF-8 it would have worked... I guess the problem is that JetBrains is generating the file in (e.g.) Windows-1252, and Python needs to be told that? Does it work if you set the environment variable PYTHONENCODING to cp1252? (I suppose I should either contact the author, or try it myself...)

Normally I'd agree, a windows-1252 misencode should be one's default guess when mojibake is afoot. Unfortunately, the errant byte in the error is 0xb3, which is "³" in windows-1252.

If you Google, "Mikołaj",

> Mikołaj is the Polish cognate of given name Nicholas

Then Google, "windows character encoding polish"

> Windows-1250 - Wikipedia

And 0xb3 is "ł" in that encoding.¹

> Does it work if you set the environment variable PYTHONENCODING (sic) to cp1252?

I don't know if setting PYTHONIOENCODING would work here; I don't think it should affect this. Really, fixing the YAML file is the fix. (And fixing the thing that generated it.)

¹it is queries like this that really make me love the search engines of today. This would have been hell in the days of Alta Vista.

Re: I couldn't debug the code because of my name

#199

Post by stubish » Thu, Oct 21, 2021, 3:47 AM UTC

geofft wrote: post #178

nightpool wrote: post #143

Earlier quoted context omitted.

The incorrect docker-compose file was generated by Java (Jetbrains) but consumed by Python (docker-compose). The GP comment was complaining about Python's strict Unicode consumption, not Java's invalid Unicode generation.

Oh, I see. But if it was UTF-8 it would have worked... I guess the problem is that JetBrains is generating the file in (e.g.) Windows-1252, and Python needs to be told that? Does it work if you set the environment variable PYTHONENCODING to cp1252? (I suppose I should either contact the author, or try it myself...)

JetBrains is generating an invalid YAML file, which are UTF-8. If they were using a decent YAML library, it would have crashed at that point. And firmly pointed the finger at the real bug, reading raw bytes from the environment or a .properties file parser and assuming it is valid UTF-8.

And this is why you always validate your data when you slurp it in, or else you pass crap down several layers where it crashes or mostly works with the potential for security holes or catastrophic behavior, and a pain in the arse to track down since the actual bug is nowhere near where you are looking.

Re: I couldn't debug the code because of my name

#200

Post by TechieKid » Thu, Oct 21, 2021, 3:53 AM UTC

aidenn0 wrote: post #94

munk-a wrote: post #92

Earlier quoted context omitted.

It's also important to width-test fields. Never forget to make sure that WWWWWWWWWWWW doesn't cause weird application wrapping.

I used a system where the maximum length on the "new password" field in the change password form was longer than the password field in the login form. The symptom was that I could login if I used my password manager browser plugin, but not if I pasted it from my password manager.

Discover (discover.com) currently has a similar bug where it'll allow me to login with my password, but will not accept the same password in the 'Change password' workflow as the old password, complaining about it being invalid. (shrug)