Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Full Reddit Submission Corpus now available for 2006 thru August 2015

reddit.com

Post reply on HN
Print view View original

81–90 of 100 posts

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#81

Post by ebbv »

Karunamon wrote: post #69
ebbv wrote: post #51

Earlier quoted context omitted.

> Reddit 2006 would not reflect mainstream US values and opinions, Reddit 2015 would be much closer. While I agree Reddit 2015 does reflect more of the US mainstream, it is still far from being an actual reflection of popular US opinion. Reddit is far more atheist/libertarian than the US mainstream. And I would argue it is right now far more reactionary/right wing as well. It has become very popular among young white…

generally retrograde attitudes towards women of Reddit in general That's a pretty awful accusation to level at ~20 million people. I wonder if you can substantiate that?

Are you kidding me? Have you ever read any comments or half the submissions on the site? Or seen tons of popular subs like Mens Rights and shit?

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#82

Post by ebbv »

imgabe wrote: post #75
archagon wrote: post #71

Earlier quoted context omitted.

I think anyone who reads the default subs can corroborate the rampant sexism and racism. When I'm on women-centric subs like /r/askwomen, I frequently see comments that would absolutely get torn apart in most other parts of the site. Not offensive or provocative ones, either. Just normal conversation that would get a slew of "shut up, SJW" anywhere else.

Considering not all of the user base comments and that of those who do, not all of them are misogynistic, imputing the opinions of a vocal minority to the whole community isn't really fair.

By definition the people who don't comment don't contribute to the opinions on the site, so your argument that they are nice people really doesn't matter, even if it were true.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#83

Post by Springtime »

nbadg wrote: post #77
Houshalter wrote: post #11

This is not as cool as the comment dataset, but it's still pretty cool. I think in the future AIs will make extensive use of these kinds of datasets. It's amazing how much useful information is in there. Last year I made a simple IRC question answering bot that just searched reddit for your question and returned the top comment of the top result. It worked surprisingly well. It's amazing how many questions someone el…

Do you have any of the source on Github? I'd be very interested in taking a look and potentially using it as an excuse to get playing with a natural language toolkit (in all my spare time haha)

Found a [1] pastebin of the code the author made in July when I believe it was updated due to interest from another HN discussion [2].

[1] http://pastebin.com/CM9u17jq

[2] https://news.ycombinator.com/item?id=9871603

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#84

Post by ebbv »

elbear wrote: post #74
ebbv wrote: post #64

Earlier quoted context omitted.

It appears you didn't understand the sentence you are partially quoting. Reddit itself is full of porn. Much like attitudes towards women are likely to be generally more misogynistic among the patrons of a strip club than they are, say, inside of a planned parenthood, not just the presence but the omnipresence of porn on Reddit, and the role the porn plays in Reddit's popularity has an effect on the site and its user…

I think your post would have gained more without the personal attack. I understand that his personal triggered your response, but the first part of your comment does a good job of supporting your argument.

I honestly don't care how my response gets voted. I responded to him much nicer and with less personal attacks than he did (I accused him of not reading/understanding my comment, and of attacking me, both of which are patently true, he accused me of being a bigot.)

Some people on HN just want you to lay there and take it when someone attacks you, and I won't. So, I get down voted when I respond to people who are nasty. It's worth it.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#85

Post by imgabe »

ebbv wrote: post #82
imgabe wrote: post #75

Earlier quoted context omitted.

Considering not all of the user base comments and that of those who do, not all of them are misogynistic, imputing the opinions of a vocal minority to the whole community isn't really fair.

By definition the people who don't comment don't contribute to the opinions on the site, so your argument that they are nice people really doesn't matter, even if it were true.

If you go to the comment section of any major site you'll find racism and misogyny. Saying Redditors support misogyny because there are misogynistic comments there is like saying New York Times readers do the same, because they also have misogynistic comments on their site.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#86

Post by Karunamon »

ebbv wrote: post #81
Karunamon wrote: post #69

Earlier quoted context omitted.

generally retrograde attitudes towards women of Reddit in general That's a pretty awful accusation to level at ~20 million people. I wonder if you can substantiate that?

Are you kidding me? Have you ever read any comments or half the submissions on the site? Or seen tons of popular subs like Mens Rights and shit?

Popular is a pretty relative term. MensRights has ~120K subscribers, while TwoXChromosomes has ~3.4 million. The latter is a lot more indicative of what the site "likes" than the former.

Anecdotes are not data.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#87

Post by Karunamon »

archagon wrote: post #71
Karunamon wrote: post #69

Earlier quoted context omitted.

generally retrograde attitudes towards women of Reddit in general That's a pretty awful accusation to level at ~20 million people. I wonder if you can substantiate that?

I think anyone who reads the default subs can corroborate the rampant sexism and racism. When I'm on women-centric subs like /r/askwomen, I frequently see comments that would absolutely get torn apart in most other parts of the site. Not offensive or provocative ones, either. Just normal conversation that would get a slew of "shut up, SJW" anywhere else.

That last comment is not necessarily anything-ist. Someone who reads a community centralized around the finder points of, say, raising kittens, probably doesn't desire social politics and its associated toxicity injected. (Imagine this conversation we're having here, right now, in the middle of your favorite place to relax, and talk to people, except a lot less civil, and that's before the meta communities find it and invade, and even if you avoid it entirely, everyone involved gets hacked off at each other because the very nature of the topic is divisive)

It's off-topic meta noise at best and harmful to the community at worst.

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#88

Post by Houshalter »

nbadg wrote: post #77
Houshalter wrote: post #11

This is not as cool as the comment dataset, but it's still pretty cool. I think in the future AIs will make extensive use of these kinds of datasets. It's amazing how much useful information is in there. Last year I made a simple IRC question answering bot that just searched reddit for your question and returned the top comment of the top result. It worked surprisingly well. It's amazing how many questions someone el…

Do you have any of the source on Github? I'd be very interested in taking a look and potentially using it as an excuse to get playing with a natural language toolkit (in all my spare time haha)

Yes https://github.com/Houshalter/amabot

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#89

Post by Houshalter »

bcruddy wrote: post #54
Houshalter wrote: post #11

This is not as cool as the comment dataset, but it's still pretty cool. I think in the future AIs will make extensive use of these kinds of datasets. It's amazing how much useful information is in there. Last year I made a simple IRC question answering bot that just searched reddit for your question and returned the top comment of the top result. It worked surprisingly well. It's amazing how many questions someone el…

This is really cool, is the source code available anywhere?

yes https://github.com/Houshalter/amabot

Re: Full Reddit Submission Corpus now available for 2006 thru August 2015

#90

Post by samstave »

TorKlingberg wrote: post #13

I wonder if an analysis of this and the comment corpus could quantifiably show shifts in the opinions of the Reddit "hivemind" since the start. Has the Reddit userbase turned more conservative over the years, or is that just my impression?

Can you take a a sentiment of certain trends in what people are talking about regarding [subject] and turn it into either better marketing or profiting off sentiments in the market?
Post reply on HN