Live data from Hacker News

Edge cases to keep in mind when working with text

thedroidsonroids.com

21–30 of 31 posts

Re: Edge cases to keep in mind when working with text

#21
post #11

Earlier quoted context omitted.

I'm not sure I understand. Are you saying that, because a misunderstanding over text led to one person murdering another, that the culture in which these events occurred was primitive, backwards, and deserving of the epithet "scum"?

Yes.

This is an instructive escalation.

User ewjordan's original comment is clearly calling out a bad institution-- an "honor-based patriarchal culture"-- for encouraging a specific bad actor (or at least a specific class of bad actors) to murder other human beings. I say bad actor because ewjordan saw fit to emphasize-- using the heightened language "damn well"-- that everyone knows about the potential for ambiguity in the choice of text. People who jump on an ambiguity in a patriarchal society for a chance to do murder are clearly bad actors. Hence, the word scum probably seems like a fitting epithet for such a bad actor.

The problem is that heightened language always ends up obscuring one's point. For example, is user dvfjsdhgfv agreeing with ewjordan or extending the critique more broadly to include people who aren't obviously bad actors? It isn't clear to me.

User Kenji's logic is less ambiguous, however. Since it is "their culture" which "causes causes behaviour you'd expect of savages a couple thousand years ago", the use of the word "scum" is deemed appropriate. Yet, using "scum" to describe a "culture" is awkward, and seems to suggest that it is the inhabitants of that culture who are in fact being called "scum" here. Does that include both bad faith and good faith actors? User jballanc asks for clarification on this, and by now user lawly is apparently confident enough to give a one-word answer: "Yes." Finally, just to confirm that the understanding is that we're indeed applying the descriptor "scum" to large swaths of human beings, user jballanc gives the counter-argument based on that logic:

> I mean, I'm no fan of the British, but I wouldn't call them "scum".

This is why I try to avoid using heightened language especially when describing people who commit horrific acts.

Re: Edge cases to keep in mind when working with text

#22

I wish more software developers kept these things in mind. At one of my customers I worked on interfacing their online store with several other software components. The store was the only piece able to handle the names of customers (from different parts of the world) correctly. All the rest failed at some point. There are so many additional aspects you discover only when you actually work on these things.

My first name is hyphenated. I run into online forms all the time that don't accept my name. Or won't accept it because it is "too long."

Re: Edge cases to keep in mind when working with text

#23
post #20

See also: ordering text with numbers in them. I've worked with several programs that present results as "Thing number 1" "Thing number 10" "Thing number 2" ... Which IS of course alphabetical order, but it is also the wrong order for any user-facing list. This is most likely a symptom of prematurely string-ifying and passing around information as strings.

Google, apparently the smartest tech company in the whole wide world, still hasn't figured this out for drive

Re: Edge cases to keep in mind when working with text

#24
post #19
post #5

The Turkish situation referenced ( http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two... ) is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. "Ramazan went to the family's home to apologize, only to be greeted by the father, Emine, two sisters and a lot of very sharp knives." There's no technological way to fix people that would try to kill someone over a text misunderstan…

OK, the overreaction like this may be a cultural problem. But even without a problem like that, imagine a case when a wrong drug is administered to a patient, with deadly consequences, or a wrong turn is taken by a motorist, leading to a collision. To avoid this, things should be written in an unequivocal way. But for that, one has to realize how expensive a "negligibly small" mistake can be.

To be fair, though, if you have two drugs that are separated only by very similar looking characters, that's a recipe for disaster anyway. Even ignoring the localization aspects, I wouldn't want two drugs separated only by the difference between i/l, u/v, etc.

Re: Edge cases to keep in mind when working with text

#25
post #20

See also: ordering text with numbers in them. I've worked with several programs that present results as "Thing number 1" "Thing number 10" "Thing number 2" ... Which IS of course alphabetical order, but it is also the wrong order for any user-facing list. This is most likely a symptom of prematurely string-ifying and passing around information as strings.

Alphabetical order is annoying to get right, and really hard if you try to apply it strictly because you have to recognize Roman numerals as numbers.

Re: Edge cases to keep in mind when working with text

#26
post #20

See also: ordering text with numbers in them. I've worked with several programs that present results as "Thing number 1" "Thing number 10" "Thing number 2" ... Which IS of course alphabetical order, but it is also the wrong order for any user-facing list. This is most likely a symptom of prematurely string-ifying and passing around information as strings.

Raymond Chen's Old New Thing covered a similar issue, contrasting NTFS vs. Windows Explorer:

https://blogs.msdn.microsoft.com/oldnewthing/20050617-10/?p=...

He referenced Michael Kaplan's MSDN blog about how Windows Explorer sorts, which took me a while to find:

http://archives.miloush.net/michkap/archive/2006/09/30/77834...

And there is a KB article about differences between Windows versions:

https://support.microsoft.com/en-us/help/319827/the-sort-ord...

The function is available for use, StrCmpLogicalW:

https://msdn.microsoft.com/en-us/library/windows/desktop/bb7...

Re: Edge cases to keep in mind when working with text

#27
post #19
post #5

The Turkish situation referenced ( http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two... ) is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. "Ramazan went to the family's home to apologize, only to be greeted by the father, Emine, two sisters and a lot of very sharp knives." There's no technological way to fix people that would try to kill someone over a text misunderstan…

OK, the overreaction like this may be a cultural problem. But even without a problem like that, imagine a case when a wrong drug is administered to a patient, with deadly consequences, or a wrong turn is taken by a motorist, leading to a collision. To avoid this, things should be written in an unequivocal way. But for that, one has to realize how expensive a "negligibly small" mistake can be.

> ...imagine a case when a wrong drug is administered to a patient, with deadly consequences...

Out of band solutions are used to address these problems more comprehensively.

The FDA regulates brand names for drugs for exactly this reason: avoid confusion and ambiguities.

This is also one of the reasons for pharmacists, especially in hospitals. Mouse-clicking in a pull down list can easily select the wrong drug for the patient.

Re: Edge cases to keep in mind when working with text

#28

I wish more software developers kept these things in mind. At one of my customers I worked on interfacing their online store with several other software components. The store was the only piece able to handle the names of customers (from different parts of the world) correctly. All the rest failed at some point. There are so many additional aspects you discover only when you actually work on these things.

My first name is hyphenated. I run into online forms all the time that don't accept my name. Or won't accept it because it is "too long."

Indeed when Brian Fox (of bash fame) implemented the name field in finger he increased it specifically so my entire name would fit.

(After I married his effort was wasted)

Re: Edge cases to keep in mind when working with text

#29
post #3

Earlier quoted context omitted.

Wouldn't you want to call that "string" versus "bytes" (instead of "text" versus "string")? (That's the Python parlance, if I'm not mistaken, and it seems good to me.)

To me , Bytes are a different abstraction. In terms of automata, a string consists of characters. Related to the problem of conflating text and strings is the traditional conflation of characters and bytes via ASCII. Again for me, in terms of handling text versus strings, the Python ecosystem is not the paragon of well considered language design. Abstractions over strings, text, characters, and bytes are a significan…

> traditional conflation of characters and bytes

But that's precisely what "string" vs "bytes" untangles... (talking Python 3 here; Python 2 suffers precisely from the mess you mention)

Re: Edge cases to keep in mind when working with text

#30
post #29

Earlier quoted context omitted.

To me , Bytes are a different abstraction. In terms of automata, a string consists of characters. Related to the problem of conflating text and strings is the traditional conflation of characters and bytes via ASCII. Again for me, in terms of handling text versus strings, the Python ecosystem is not the paragon of well considered language design. Abstractions over strings, text, characters, and bytes are a significan…

> traditional conflation of characters and bytes But that's precisely what "string" vs "bytes" untangles... (talking Python 3 here; Python 2 suffers precisely from the mess you mention)

The previous comment used "Python" without specifying which one...and part of me feels that to some degree Python 3 just put another layer on the mess because it still did not make the important distinction between texts as human readable artifacts and strings as a computer science concept. Or to put it another way, a stream of bytes can readily be parsed as a string of characters by an automaton because bytes can be parsed as characters and a stream as a string. But this all happens at a lower level of abstraction than text as something that humans read and write both with and without computers. I mean Python 3's distinction between strings and bytes in no way address messy problems like ß/ss in German.
Post reply on HN