Live data from Hacker News

Full Reddit Submission Corpus now available for 2006 thru August 2015

reddit.com

91–100 of 100 posts

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#91
post #77

Earlier quoted context omitted.

Do you have any of the source on Github? I'd be very interested in taking a look and potentially using it as an excuse to get playing with a natural language toolkit (in all my spare time haha)

Yes https://github.com/Houshalter/amabot

Thanks man!

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#92

Earlier quoted context omitted.

I think anyone who reads the default subs can corroborate the rampant sexism and racism. When I'm on women-centric subs like /r/askwomen, I frequently see comments that would absolutely get torn apart in most other parts of the site. Not offensive or provocative ones, either. Just normal conversation that would get a slew of "shut up, SJW" anywhere else.

That last comment is not necessarily anything-ist. Someone who reads a community centralized around the finder points of, say, raising kittens, probably doesn't desire social politics and its associated toxicity injected. (Imagine this conversation we're having here, right now, in the middle of your favorite place to relax, and talk to people, except a lot less civil, and that's before the meta communities find it an…

Well, the fact of the matter is, most Reddit communities are dominated by male voices. So the "social politics" often aren't artificial incursions into the discussion, but underrepresented groups just trying to be part of the conversation. For example, there was recently some controversy about the portrayal of a female character in Metal Gear Solid V. It's not good when a woman says "I don't like how this character was portrayed" and everyone else pipes in with "lol SJW feminist". It's just a shitty, shitty atmosphere. In contrast, I've found that in communities with a more equal gender split, benefit of the doubt is far more freely given, and some great discussions about potentially touchy topics happen as a result.

Anyway, I've found that the people who use shorthand insults like "SJW" are typically part of the entrenched majority group of their subculture — and want nothing more than to keep those pesky "outsiders" away from their clubhouse. (Despite the fact that those outsiders had, in fact, been there all along.)

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#93

Earlier quoted context omitted.

That last comment is not necessarily anything-ist. Someone who reads a community centralized around the finder points of, say, raising kittens, probably doesn't desire social politics and its associated toxicity injected. (Imagine this conversation we're having here, right now, in the middle of your favorite place to relax, and talk to people, except a lot less civil, and that's before the meta communities find it an…

Well, the fact of the matter is, most Reddit communities are dominated by male voices. So the "social politics" often aren't artificial incursions into the discussion, but underrepresented groups just trying to be part of the conversation. For example, there was recently some controversy about the portrayal of a female character in Metal Gear Solid V. It's not good when a woman says "I don't like how this character w…

Personally, I'd love to know what communities those are. I've yet to find any one that would take that comment you just used as an example (or its converse) and allow anyone else to disagree with it freely. (And when I say freely, I mean without namecalling, without mass downvotes, without banning dissent, without any of the other community-forcing-its-will-on-the-minority-opinion garbage that follows those discussions 99% of the time in my experience)

"I don't like how this character was portrayed" is a fine criticism on its own, but behind that is the shadow of "..so what should be done about it?". Absolutely nothing of value lays down that path.

Put another way: Talk of frame rates leads to discussions about optimization. Talks of nonsensical writing leads to discussions of something else they could have done. Talks about portrayal of a "minority" character go straight into "the developers are *ist".

Why does "being part of the conversation" always seem to wind up in "I don't like this, the devs are assholes, it should be changed?"

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#94
post #51

Earlier quoted context omitted.

> Reddit 2006 would not reflect mainstream US values and opinions, Reddit 2015 would be much closer. While I agree Reddit 2015 does reflect more of the US mainstream, it is still far from being an actual reflection of popular US opinion. Reddit is far more atheist/libertarian than the US mainstream. And I would argue it is right now far more reactionary/right wing as well. It has become very popular among young white…

> young white males who have unhealthy attitudes towards women due to the abundance of porn You sound just like the fundamentalist Evangelicals I grew up with.

> You sound just like the fundamentalist Evangelicals I grew up with.

That reads as if you intended it as a personal attack. Such comments are not welcome on Hacker News. Please don't post them here.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#95

I wonder if an analysis of this and the comment corpus could quantifiably show shifts in the opinions of the Reddit "hivemind" since the start. Has the Reddit userbase turned more conservative over the years, or is that just my impression?

You could probably just take the same methodology as the Poole-Rosenthal index for partisan politics in the United States. It's always the most widely cited for the current state of divisiveness in politics and I think could be applied here.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#96

I wonder if an analysis of this and the comment corpus could quantifiably show shifts in the opinions of the Reddit "hivemind" since the start. Has the Reddit userbase turned more conservative over the years, or is that just my impression?

You could probably just take the same methodology as the Poole-Rosenthal index for partisan politics in the United States. It's always the most widely cited for the current state of divisiveness in politics and I think could be applied here.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#97
Crossref did some analysis on when DOIs are submitted to reddit. DOIs are identifiers / persistent links, mostly for scholarly content.

http://crosstech.crossref.org/2015/09/dois-in-reddit.html

HN thread here: https://news.ycombinator.com/item?id=10303295

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#98
post #77

Earlier quoted context omitted.

Do you have any of the source on Github? I'd be very interested in taking a look and potentially using it as an excuse to get playing with a natural language toolkit (in all my spare time haha)

Yes https://github.com/Houshalter/amabot

Could you share "C:\\Users\\Daniel\\Documents\\Programming\\AMAbotData.txt" as well or it is not neccessary to run the bot?

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#99

Earlier quoted context omitted.

Yes https://github.com/Houshalter/amabot

Could you share "C:\\Users\\Daniel\\Documents\\Programming\\AMAbotData.txt" as well or it is not neccessary to run the bot?

Presumably that's the large dataset he was referring to.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#100
post #99

Earlier quoted context omitted.

Could you share "C:\\Users\\Daniel\\Documents\\Programming\\AMAbotData.txt" as well or it is not neccessary to run the bot?

Presumably that's the large dataset he was referring to.

Ok I added it and removed some of the stuff that was configured specifically for me. It should work. Please report any and all issues. I'm not sure if javascript supports local filepaths so you might need to edit it.
Post reply on HN