Live data from Hacker News

Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

blog.effectcheck.com

41–50 of 57 posts

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#41
post #38
post #35

Earlier quoted context omitted.

Check out the results for their own webpage. Click "Try EffectCheck on this page!" here: http://effectcheck.com/tour I'm a little skeptical. It rates "anxiety" as "medium-high." Yet I don't feel the slightest bit anxious reading that page. Nor does the author come off as anxious. It all feels very positive to me. (I'm counting the text alone, not the graphic design.) It rates "compassion" as "very high." I wasn't abl…

>It's also possible that I'm missing the point by comparing how I think I feel against EffectCheck's results. Maybe we're unreliable observers of how a text emotionally affects us, and EffectCheck more accurately predicts what's really going on deep in our brains. In which case, forgive my unwarranted naysaying. Bingo. Our algorithm focuses on the lingering, subliminal emotions that you aren't necessarily aware you'r…

How can we falsify your claims? That is, what is a test we can perform that if it went a certain way, would show that your claims are false?

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#42
In case anyone wants to make their own trends (less sophisticated, but maybe more transparent), I have an app at http://hn-trends.heroku.com that plots word percentages on HN over time.

For example, some emotive words that appear to have increased since the beginning:

* sad: http://hn-trends.heroku.com/trends?q=sad

* fuck: http://hn-trends.heroku.com/trends?q=fuck

* shit: http://hn-trends.heroku.com/trends?q=shit

Some words that appear to have decreased:

* great: http://hn-trends.heroku.com/trends?q=great

* cool: http://hn-trends.heroku.com/trends?q=cool

* lol: http://hn-trends.heroku.com/trends?q=lol

* reddit: http://hn-trends.heroku.com/trends?q=reddit

* digg: http://hn-trends.heroku.com/trends?q=digg

I haven't looked at any of this closely, so make of it what you will.

(I was meaning to wait until I had time to dig deeper and add extra features like time series smoothers and trend/slope metrics before "releasing" this, but figured this might be useful now given the post. I still plan on adding those later.)

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#43
post #7

I don't mean to come across as hostile or undermine the effort that's been invested, but I can't help but be a tad skeptical about these visualizations, especially given the fact that the author (in what might be construed as cavalier fashion) presents seemingly nebulous metrics like they are absolute matter-of-fact ("Anxiety/Confidence Ratio", "Hostility/Compassion Ratio", "Depression/Happiness Ratio"). It would cer…

>And with that, I throw down the gauntlet. Analyze this! Sure. Scored with EffectCheck: Anxiety - Very High Hostility - High Depression - Very High Confidence - Low Compassion - Low Happiness - Very Low I will post a longer explanation detailing how/why it works, since others have had this question as well.

Whats your model for folks who have proven mental health issues? I have little faith that a clinically depressed, or worse, person would fit the same profile.

And there's got to be enough folks with mental health problems to destroy your margin of error, this is the lonely internet afterall.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#44
post #38
post #35

Earlier quoted context omitted.

Check out the results for their own webpage. Click "Try EffectCheck on this page!" here: http://effectcheck.com/tour I'm a little skeptical. It rates "anxiety" as "medium-high." Yet I don't feel the slightest bit anxious reading that page. Nor does the author come off as anxious. It all feels very positive to me. (I'm counting the text alone, not the graphic design.) It rates "compassion" as "very high." I wasn't abl…

>It's also possible that I'm missing the point by comparing how I think I feel against EffectCheck's results. Maybe we're unreliable observers of how a text emotionally affects us, and EffectCheck more accurately predicts what's really going on deep in our brains. In which case, forgive my unwarranted naysaying. Bingo. Our algorithm focuses on the lingering, subliminal emotions that you aren't necessarily aware you'r…

That seems very believable in a general sense, and I have vague memories of other studies that suggest similar things.

The remaining question for me is whether and how EffectCheck's results have been compared against actual measurements of these subliminal effects.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#45
post #39

"Thus, if we look at the PG analysis as a measure of his mental state, we can speculate this is due to the prosperity he’s experienced in these last few years." Actually it's because the site has become so large that I have to be more diplomatic. When it was small I could say harsh-sounding things without worrying about being misinterpreted. Now I have to be more of a politician.

That was my other hypothesis, but it didn't really correlate well with the growth in the site's user base. HN has experienced an exponential growth rate, but your comments show a linear decline in negative impact. I went with the "better life" theory because it seems like personal prosperity and happiness would be more likely to show gradual, linear change. This is also supported by the trends I saw in other famous top HN users, particularly those that I know have experienced increased financial prosperity.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#46
Others have done a good job at critiquing the automated analysis, so I will concentrate on the hypothesis.

>There is a perceived phenomenon among online social news communities (e.g. Digg, reddit) that as the popularity of the community increases, the average member becomes baser and the overall quality of discourse decreases.

I think that's not an accurate portrayal. Having been through the "decline" of Reddit (been reading it even before it had comments!), I observed that the real decline was in the quality(as perceived by me) of stories that made the frontpage(s) and in the quality(as perceived by me) of comments that were upvoted.

In 2006, I would literally read almost every story on the first few pages and the content was very interesting, there were long essays and articles that were on the front page. Now? After the influx of the general public and the juveniles(not saying this in a demeaning manner), quality has dropped in every way. Pun and argumentative threads were the ones that were on top. I do like humor but not just endless pages of snark and humor. Pictures or story-in-headline posts requiring an attention span of 3 seconds tend to get the most votes,

>If a community were suffering from SND, there are a few symptoms we might observe:

>Anxiety, hostility, and depression would rise.

>Confidence, compassion, and happiness would fall.

>The ratios of anxiety/hostility, hostility/compassion, depression/happiness would rise.

>Since each of the negative emotions has an opposite emotion, the last point enables us to measure the general negative/positive trend in each of the three main categories.

How does this analysis address the above points? All the supposed-to-be-funny short comments probably would count as a positive under this analysis. Reddit's community now is pretty happy, confident and compassionate. The analysis is pretty lacking when applied to a supposed-to-be intellectually stimulating social site.

Anyway I see that Slashdot(although not as popular now) and HN seem to have bucked the trend. Although there is some decline on HN, it's not as as steep or as bad as Reddit so far.

Edit: Here's an idea, do your analysis of all the comments of the first x posts of Reddit vs. the same on HN. Could be interesting since they both have almost the exact same format.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#47
post #45
post #39

"Thus, if we look at the PG analysis as a measure of his mental state, we can speculate this is due to the prosperity he’s experienced in these last few years." Actually it's because the site has become so large that I have to be more diplomatic. When it was small I could say harsh-sounding things without worrying about being misinterpreted. Now I have to be more of a politician.

That was my other hypothesis, but it didn't really correlate well with the growth in the site's user base. HN has experienced an exponential growth rate, but your comments show a linear decline in negative impact. I went with the "better life" theory because it seems like personal prosperity and happiness would be more likely to show gradual, linear change. This is also supported by the trends I saw in other famous t…

Instant link bait if you let us look up our own charts.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#48
post #31
post #7

Earlier quoted context omitted.

>And with that, I throw down the gauntlet. Analyze this! Sure. Scored with EffectCheck: Anxiety - Very High Hostility - High Depression - Very High Confidence - Low Compassion - Low Happiness - Very Low I will post a longer explanation detailing how/why it works, since others have had this question as well.

I'm not sure that I understand. You have just demonstrated that the algorithm does not work.

What this kind of analysis needs is a blind test. Ask three groups to rate text independently in the same metrics - 1) General public. 2) Psychologists 3) Algorithm.

The results should be interesting. However, if the authors claim subliminal effects, then I do not how that could be tested.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#49

Certainly we can all argue about the underlying structure of the analysis of the data. However, I think that @tansey has done two great services here: 1.He's proposed a full service taxonomy with by proposing a name for the phenomenon SND. Which is much a much better choice than JTS - Jumping the Shark or otherwise. 2. He's asking how we can evaluate that phenomenon. Proposing one solution. So the question becomes, h…

> 5. More community centered posts aka AMA etc.

I consider AMA the best kind of self post... I don't see community centered posts as necessarily a bad thing except if there is too much of drama or he-did, she-did type content.

Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)

#50
post #37

Earlier quoted context omitted.

This is excellent. Would you be willing to check this text from my blog please? "It's said that great military commanders, chess players and Go players feel physical pressure on their stomachs when their game pieces are threatened, and the pressure indicates the moves to make. This full-body thinking communicates much more rapidly than purely deductive mental reasoning. The intuition is the result of thousands of pri…

Sure. Here you go: Anxiety - High Hostility - High Depression - High Confidence - Medium Compassion - Very Low to Low Happiness - Low to Medium

Haha, thanks. Looking forward to an explanation of this.
Post reply on HN