Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Edge cases to keep in mind when working with text

thedroidsonroids.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Edge cases to keep in mind when working with text

#11

Post by lawly »

jballanc wrote: post #10
Kenji wrote: post #9

Earlier quoted context omitted.

> Calling people for their culture "scum" is the least effective way of "positively" influencing them If their culture is objectively so primitive and backwards that it causes behaviour you'd expect of savages a couple thousand years ago, I think that word is entirely appropriate.

I'm not sure I understand. Are you saying that, because a misunderstanding over text led to one person murdering another, that the culture in which these events occurred was primitive, backwards, and deserving of the epithet "scum"?

Yes.

Re: Edge cases to keep in mind when working with text

#12

Post by jballanc »

lawly wrote: post #11
jballanc wrote: post #10

Earlier quoted context omitted.

I'm not sure I understand. Are you saying that, because a misunderstanding over text led to one person murdering another, that the culture in which these events occurred was primitive, backwards, and deserving of the epithet "scum"?

Yes.

I mean, I'm no fan of the British, but I wouldn't call them "scum".

Oh, or did you mean a different situation where a misunderstanding via text led to a murder? http://www.dailymail.co.uk/news/article-1356632/Man-killed-f...

Re: Edge cases to keep in mind when working with text

#13

Post by lawly »

jballanc wrote: post #12
lawly wrote: post #11

Earlier quoted context omitted.

Yes.

I mean, I'm no fan of the British, but I wouldn't call them "scum". Oh, or did you mean a different situation where a misunderstanding via text led to a murder? http://www.dailymail.co.uk/news/article-1356632/Man-killed-f...

Oh, yes, let's pretend it's the same thing when a nutjob commits manslaughter and a family comes together over knives to commit premeditated murder of a man coming to apologize to defend the honor of the family's daughter.

Re: Edge cases to keep in mind when working with text

#14

Post by the8472 »

rocqua wrote: post #6
FabHK wrote: post #3

Earlier quoted context omitted.

Wouldn't you want to call that "string" versus "bytes" (instead of "text" versus "string")? (That's the Python parlance, if I'm not mistaken, and it seems good to me.)

In that case, which is HTML? Because it's meant for machines to read, so should be treated more carefully that pure 'text'. However, HTML shouldn't be processed like bytes either. I suppose with HTML the real issue is that there is human readable text in there.

HTML5 is specified in terms of unicode characters and forbids most control and non-characters[0]. In that sense it is human-readable text. It's not easy to read, especially when minified. But so is shorthand or very dense scientific texts.

Some things are meant to be human-readable and machine-parsable at the same time. Those things are not mutually exclusive.

And sometimes humans will complain even more than machines if you mangle their text in any way.

[0] https://html.spec.whatwg.org/#preprocessing-the-input-stream

Re: Edge cases to keep in mind when working with text

#15

Post by dvfjsdhgfv »

jlg23 wrote: post #8
ewjordan wrote: post #5

The Turkish situation referenced ( http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two... ) is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. "Ramazan went to the family's home to apologize, only to be greeted by the father, Emine, two sisters and a lot of very sharp knives." There's no technological way to fix people that would try to kill someone over a text misunderstan…

> is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. Bad tech played a crucial role in the escalation, though. What is justified in one society or even demanded is not necessarily justified in another. And a text message that you consider just offensive might be a strong insult in another culture with implications you cannot even grasp. Calling people for their culture "scum" is the…

How on earth should people who kill over words, no matter how offensive, be called? Really, in these times of twisting the meaning of words, we should really call spade a spade and not to hide behinds untrue cliches such as "all cultures are equal and should be respected equally". Not they're not and shouldn't be. If your "culture" demands that you pour acid over the face of your ex-girlfriend, it doesn't deserve to be called "culture", and you deserve to be called who you are.

Re: Edge cases to keep in mind when working with text

#16

Post by brudgers »

FabHK wrote: post #3
brudgers wrote: post #2

Text (aka strings) exists in virtually all software projects For me, distinguishing between text as something that is intended to be read by humans and strings as serial sequences of characters that may or may not be human readable but will be processed by one or more computing automata is useful. For example in C, the string "Hello World" is terminated by a null character. The null character is not part of the text…

Wouldn't you want to call that "string" versus "bytes" (instead of "text" versus "string")? (That's the Python parlance, if I'm not mistaken, and it seems good to me.)

To me, Bytes are a different abstraction. In terms of automata, a string consists of characters. Related to the problem of conflating text and strings is the traditional conflation of characters and bytes via ASCII.

Again for me, in terms of handling text versus strings, the Python ecosystem is not the paragon of well considered language design. Abstractions over strings, text, characters, and bytes are a significant breaking change between Python 2 and 3 and among the reasons that Python 2 code is still written in significant amounts...and still there is no 'text' type.

Re: Edge cases to keep in mind when working with text

#17

Post by [deleted] »

FabHK wrote: post #3
brudgers wrote: post #2

Text (aka strings) exists in virtually all software projects For me, distinguishing between text as something that is intended to be read by humans and strings as serial sequences of characters that may or may not be human readable but will be processed by one or more computing automata is useful. For example in C, the string "Hello World" is terminated by a null character. The null character is not part of the text…

Wouldn't you want to call that "string" versus "bytes" (instead of "text" versus "string")? (That's the Python parlance, if I'm not mistaken, and it seems good to me.)

[deleted]

Re: Edge cases to keep in mind when working with text

#18

Post by jlg23 »

dvfjsdhgfv wrote: post #15
jlg23 wrote: post #8

Earlier quoted context omitted.

> is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. Bad tech played a crucial role in the escalation, though. What is justified in one society or even demanded is not necessarily justified in another. And a text message that you consider just offensive might be a strong insult in another culture with implications you cannot even grasp. Calling people for their culture "scum" is the…

How on earth should people who kill over words, no matter how offensive, be called? Really, in these times of twisting the meaning of words, we should really call spade a spade and not to hide behinds untrue cliches such as "all cultures are equal and should be respected equally". Not they're not and shouldn't be. If your "culture" demands that you pour acid over the face of your ex-girlfriend, it doesn't deserve to…

I have no idea who you are responding too. Nobody in this thread claimed "all cultures are equal and should be respected equally".

I was saying that cultures are vastly different and that showing complete disrespect for something you don't even understand will not help anybody at all.

Acid-attacks, btw, are a mostly Indian specialty - try this in a honor based society and you have her family get at you sooner than you can write another uninformed "response" to something nobody ever said.

Re: Edge cases to keep in mind when working with text

#19

Post by nine_k »

ewjordan wrote: post #5

The Turkish situation referenced ( http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two... ) is not an indictment of bad tech, but of a fucked up honor-based patriarchal culture. "Ramazan went to the family's home to apologize, only to be greeted by the father, Emine, two sisters and a lot of very sharp knives." There's no technological way to fix people that would try to kill someone over a text misunderstan…

OK, the overreaction like this may be a cultural problem.

But even without a problem like that, imagine a case when a wrong drug is administered to a patient, with deadly consequences, or a wrong turn is taken by a motorist, leading to a collision.

To avoid this, things should be written in an unequivocal way. But for that, one has to realize how expensive a "negligibly small" mistake can be.

Re: Edge cases to keep in mind when working with text

#20

Post by bloaf »

See also: ordering text with numbers in them.

I've worked with several programs that present results as

"Thing number 1"

"Thing number 10"

"Thing number 2"

...

Which IS of course alphabetical order, but it is also the wrong order for any user-facing list. This is most likely a symptom of prematurely string-ifying and passing around information as strings.

Post reply on HN