Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
blog.effectcheck.com
Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
1–10 of 57 posts
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#2How does the author measure anxiety, hostility, depression, confidence, compassion, and happiness? That seems to be left out.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#3How does the author measure anxiety, hostility, depression, confidence, compassion, and happiness? That seems to be left out.
Hi! My startup [1] is based on an algorithm that can measure the emotional impact of text. Our blog is where we post interesting results from applying our algorithm.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#4How does the author measure anxiety, hostility, depression, confidence, compassion, and happiness? That seems to be left out.
Hi! My startup [1] is based on an algorithm that can measure the emotional impact of text. Our blog is where we post interesting results from applying our algorithm. [1] http://effectcheck.com
Uhhhhh-huh. But how does the algorithm work, and (more importantly) how do we know whether it works or not?
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#5How does the author measure anxiety, hostility, depression, confidence, compassion, and happiness? That seems to be left out.
Hi! My startup [1] is based on an algorithm that can measure the emotional impact of text. Our blog is where we post interesting results from applying our algorithm. [1] http://effectcheck.com
I assume the actual algorithm itself is not open sourced.
I'm wondering if there's anything that trips it up, like adding points to a comment's "hostility" score for profanities, which some people use when they're fucking excited about an idea.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#6I don't mean to come across as hostile or undermine the effort that's been invested, but I can't help but be a tad skeptical about these visualizations, especially given the fact that the author (in what might be construed as cavalier fashion) presents seemingly nebulous metrics like they are absolute matter-of-fact ("Anxiety/Confidence Ratio", "Hostility/Compassion Ratio", "Depression/Happiness Ratio").
It would certainly help if the algorithm used to compute these metrics were shared and dissected. I tend to believe that sentiment analysis is much more art than science, given how tricky and profound context can be.
To conclude, I'm not going to end this comment on a negative note. I'd much rather reserve my vitriol and caustic criticism for another thread and day and abstain from calling this an attempt at gaming HN to promote a startup.
I do this not out of a lack of indignation towards what I've just read, but because I'd like to end this seemingly hateful comment on a note that isn't bitter or negative but is instead quite the opposite (without using a single word that would help the OP's algorithm figure this out).
And with that, I throw down the gauntlet. Analyze this!
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#7I don't mean to come across as hostile or undermine the effort that's been invested, but I can't help but be a tad skeptical about these visualizations, especially given the fact that the author (in what might be construed as cavalier fashion) presents seemingly nebulous metrics like they are absolute matter-of-fact ("Anxiety/Confidence Ratio", "Hostility/Compassion Ratio", "Depression/Happiness Ratio"). It would cer…
>And with that, I throw down the gauntlet. Analyze this!
Sure. Scored with EffectCheck:
Anxiety - Very High
Hostility - High
Depression - Very High
Confidence - Low
Compassion - Low
Happiness - Very Low
I will post a longer explanation detailing how/why it works, since others have had this question as well.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#8How does the author measure anxiety, hostility, depression, confidence, compassion, and happiness? That seems to be left out.
Hi! My startup [1] is based on an algorithm that can measure the emotional impact of text. Our blog is where we post interesting results from applying our algorithm. [1] http://effectcheck.com
Trying the sample EffectCheck script on http://effectcheck.com/about : the sidebar says it's counting occurrences of phrases which score on each dimension. How is the association of word/topic determined? The measurement is valuable if and only if "Anxiety" defined as the count of ("different", "trigger", "taken out", "over", "carefully"...) corresponds to anxiety on some other psychologically relevant scale.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#9Hi everyone,
So there seems to be a lot of skepticism about whether our algorithm can actually measure emotional impact accurately. For the long answer, I'll refer you to the about page[1] for EffectCheck. For the short answer:
My co-founders are an AI PhD and a Clinical Psychology PhD. They spent three years curating a huge dictionary of words using a methodology similar to the Harvard Psychosocial Dictionary [2], but with the twist that they were focused on lexical impact of words. The dictionary is pretty accurate at measuring both impact and sentiment [3]. For example, we can predict Amazon reviews as being positive or negative, using the stars to validate if we are correct-- blog post on that coming soon.
Regarding context of the word usage: Both behavioral studies and fMRI scans have confirmed that context is not as important as one might believe. Our brains process multiple meanings of words in parallel, and the emotions associated with those words linger in our subconscious even after we know the correct context. Similarly, in cases of reviews and comments, people who use hostility-evoking words are often hostile themselves (angry people tend to make others angry) and the same is true of the other five fundamental emotions we measure.
Happy to answer more questions. Also happy to analyze any data that you would like to see in order to verify accuracy of the algorithm-- just give me a link or the text. :)
[1] http://effectcheck.com/about
[2] http://www.wjh.harvard.edu/~inquirer/homecat.htm
[3] Note that sentiment is correlated to impact only if the writer or speaker is writing without detailed attention to word choice. For example, political speech writers comb over every word to make sure they have the desired impact-- thus, the final text likely has little correlation to the sentiment of the orator.
Re: Do Social News Sites Deteriorate? (Analysis of 1.8M HN Comments)
#10Where are the confidence intervals and error bars? In order to be taken seriously when aggregating 1.8MM comments, you need those. The variability in pg's plots makes me think the data for any individual is going to be just as noisy.