Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

1.He's proposed a full service taxonomy with by proposing a name for the phenomenon SND. Which is much a much better choice than JTS - Jumping the Shark or otherwise.

2. He's asking how we can evaluate that phenomenon. Proposing one solution.

So the question becomes, how else can we evaluate the phenomenon and what can we do to reduce SND?

Well having spent time elsewhere, here are a number of clear indicators of SND:

1. Shorter, less thoughtful responses, often veering into humor or the absurd. With chuckles getting the most upvotes

2. Less fact checking and less source linking in both posts and comments

3. More image / pic posting

4. Linkjacking, with materials not linked to the original materials.

5. More community centered posts aka AMA etc.

6. Fewer news links.

So perhaps simply evaluating the length of comments of that same 1.8M HN data could support PG allegations.

The next question is what can be done to prevent SND?

I think that would be clear: Don't support the characteristics that lead to the decline.

For example, words that are generally positive may be only used in a snarky context in hacker circles, whereas words that are generally negative may similarly be used with positive affect (e.g. the word "hacker" itself). Did you customise your lexicon at all for this genre?

Also, in my experience, it's not the lexicon of words -- heck, SentiWordNet has existed for long enough -- it's what you do with that input signal that counts. Many "bag of words" approaches are on the shelf now, and very few of them are particularly accurate or clever, but most of them do the job just well enough that they get away with it.

But what tests have you done on your broader methodology? What experiments can you really do to figure out the extent to which the use of the word "nosegay" is correlated with actual depression?

Also, as someone else said, where are the error bars? If there really is a correlation between word choice and other metrics, then some simple statistics should give you error bars on your other metrics, right?

Oh, one more thing: in the example on your website you say that the sentence:

"That joke kills me!"

is "subconsciously" aggressive. My question: would your algorithm rate that at exactly the same level of aggression as the sentence:

"I'm gonna kill you!"?

cuz, y'know, intuitively one seems rather more aggressive than the other.

"It's said that great military commanders, chess players and Go players feel physical pressure on their stomachs when their game pieces are threatened, and the pressure indicates the moves to make. This full-body thinking communicates much more rapidly than purely deductive mental reasoning. The intuition is the result of thousands of prior episodes where such reasoning was employed, acted on and the outcome experienced in all its pain or glory.

Other than hours of practicing the game, or whatever one does, the only other way to improve the chance of learning this physical intuition is to be sure one's body is not sending conflicting signals. Please eat well and exercise."