Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My last name makes me invisible to Computers (2015)

wired.com

Post reply on HN
Print view View original

71–80 of 140 posts

Re: My last name makes me invisible to Computers (2015)

#71

Post by Animats »

Freak_NL wrote: post #31
tyingq wrote: post #2

I have seen apps that choke on much more common names. Like O'Brian. This post is a classic on various name issues: http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-b...

Tangentially related (rule 9); my girlfriend's surname contains an 'é'. I have yet to see a year go by without receiving mail having 'é' on the address label where the é should be. Explanation: echo "é" | iconv -t utf8 -f iso8859-15 We're Dutch, and the é is part of our language, and even part of the legacy character encoding standard everyone used before Unicode's widespread adoption. This is just a matter of code…

It's getting much better. Almost everything new is in UTF-8.

I've been writing code to clean up a 2013 database dump. The database stored everything in LATIN-1 fields. Not because the data is in LATIN-1, but because LATIN-1 will accept any byte value. This makes error messages during input go away. See this bad advice on Stack Overflow.[1]

Some of the data is ASCII. Some is UTF-8. Some is Windows-1252. Some data is none of those, but is mostly ASCII except that there's a 0x9d once in a while. (Still haven't figured out what character set that is. From context, the ™ or ® symbol is intended.) So I have recognizers for these cases, and convert everything to UTF-8, testing every field value individually.

One column has garbaged non-English names. Someone had tried to "normalize" UTF-8 to lower case by using an ASCII lowercasing function on UTF-8 stored in a LATIN-1 field:

    KACMAZLAR MEKANİK  -> kacmazlar mekanä°k
    Anita Calçados -> anita calã§ados
    Felfria Resor för att Koh Lanta -> felfria resor fã¶r att koh lanta
I have the un-"normalized" form and can fix this.

[1] https://stackoverflow.com/questions/44251813/unicodedecodeer...

Re: My last name makes me invisible to Computers (2015)

#72

Post by glic3rinu »

jmull wrote: post #39

I don't get what's happening at the low-level for this to be a problem. It seems like you'd have to do something pretty stupid at the coding level to introduce a problem with "Null" by mistake . I'm sure it happens, but not more of an occasional issue. My best guess is that there are common old databases that did not have a first class null type where it was common practice to use the string "NULL" for that purpose.…

I believe the problem might be more common when exporting data from one system to another. I've written code to migrate (and merge) large datasets from old legacy systems. I needed quite a lot of heuristics and "best-effort" transformations in order to deal with data inconsistencies... mostly string manipulations where it's not hard to imagine bugs like this happening.

Re: My last name makes me invisible to Computers (2015)

#73

Post by timcederman »

tyingq wrote: post #2

I have seen apps that choke on much more common names. Like O'Brian. This post is a classic on various name issues: http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-b...

Astonishingly, nest.com does not work for me at all if I use the hyphen in my last name.

Re: My last name makes me invisible to Computers (2015)

#74

Post by gpvos »

jamescostian wrote: post #41
kijin wrote: post #33

Earlier quoted context omitted.

The backtick and apostrophe are often used to emulate ``curly quotes,'' which makes the confusion even worse.

I think this style might come from TeX, based on this comment: https://english.stackexchange.com/questions/17695/any-refere... Manpages aren't written in TeX though (apparently they use something called "roff"), but they also contain things like `read' instead of 'read'... perhaps manpage writers tended to like TeX too?

Probably the other way round: roff/troff/nroff is older than TeX. I guess that on old computer typesetters, ` and ' turned into nicely balanced quotes.

Re: My last name makes me invisible to Computers (2015)

#75

Post by amenghra »

ajuc wrote: post #55
incongruity wrote: post #35

Earlier quoted context omitted.

Sure – that's just casting a null type as a string. So I guess if you have a comparison that somehow casts null to a string before comparing, you could run into this issue, but that's still bad programming. I used to own null@myundergrad.edu as an email alias (with my real university, not that generic .edu, of course) and I got all sorts of interesting things... but that was very intentional on my part.

Funnily enough - it's not the null-> String conversion, it's the "+" operator. That was at one point subject of heated debate in my previous job :) See: "null".equals((String)null) //false "null".equals((String)null+"") //true System.out.print((String)null) // throws NPE System.out.println((String)null) // prints "null", I guess because it appends "\n" inside

    System.out.print((String)null)
Does not throw NPE.

Re: My last name makes me invisible to Computers (2015)

#78

Post by ajuc »

amenghra wrote: post #75
ajuc wrote: post #55

Earlier quoted context omitted.

Funnily enough - it's not the null-> String conversion, it's the "+" operator. That was at one point subject of heated debate in my previous job :) See: "null".equals((String)null) //false "null".equals((String)null+"") //true System.out.print((String)null) // throws NPE System.out.println((String)null) // prints "null", I guess because it appends "\n" inside

System.out.print((String)null) Does not throw NPE.

it does here:

http://www.javarepl.com/term.html

But you're right, in regular java environment it doesn't now that I've checked.

Re: My last name makes me invisible to Computers (2015)

#79

Post by mark-r »

dsp1234 wrote: post #50
halfdgvfyjjuc wrote: post #38

Earlier quoted context omitted.

That character is a backtick not a grave isn't it? Also some cultures use «» as quotes. It's a big world.

https://en.wikipedia.org/wiki/Grave_accent "Programmers use the grave accent symbol as a separate character (i.e., not combined with any letter) for a number of tasks. In this role, it is known as a backquote or backtick."

ASCII never really supported the idea of a combining character, so it's not surprising that the original intent was subverted.
Post reply on HN