Full Reddit Submission Corpus now available for 2006 thru August 2015

Post by memeda » Mon, Sep 28, 2015, 10:49 AM UTC

TorKlingberg wrote: post #13

I wonder if an analysis of this and the comment corpus could quantifiably show shifts in the opinions of the Reddit "hivemind" since the start. Has the Reddit userbase turned more conservative over the years, or is that just my impression?

  /  l \
     /    l  \
   /      l    \
 /        l          \

Post by eru » Mon, Sep 28, 2015, 11:05 AM UTC

hippo8 wrote: post #19

tdkl wrote: post #14

Earlier quoted context omitted.

But we cannot download that one. This is actually what has been bothering me for a while now. In the past where search engines (well at least the big one) weren't that thorough, you didn't mind if something stayed on the Internet, since it wasn't easily accessible. But it's a major difference if it's indexed/searchable or downloadable.

This is why, I think we need to make ICT (Information Communication Technology) a mandatory subject in our schools. Teach kids about the internet. Many people are still under the assumption just deleting something would make it disappear for ever. Maybe a lot of people from our generation are doomed to sharing too many things online, but we can at least save the next generation from themselves. You make a mistake 10…

Post by laumars » Mon, Sep 28, 2015, 11:16 AM UTC

tomtoise wrote: post #27

Houshalter wrote: post #11

This is not as cool as the comment dataset, but it's still pretty cool. I think in the future AIs will make extensive use of these kinds of datasets. It's amazing how much useful information is in there. Last year I made a simple IRC question answering bot that just searched reddit for your question and returned the top comment of the top result. It worked surprisingly well. It's amazing how many questions someone el…

This ties directly into the comment dataset, made by the same person and via the same method[1]. Working on pulling down the submission data now, will have to spin up another server if I want to do the comment/submission correlation. >>"This dataset will go nicely with the full Reddit Comment Corpus that I released a couple months ago. The link_id from each comment corresponds to the id key in each of the submission…

Post by tomtoise » Mon, Sep 28, 2015, 11:27 AM UTC

laumars wrote: post #36

tomtoise wrote: post #27

Earlier quoted context omitted.

This ties directly into the comment dataset, made by the same person and via the same method[1]. Working on pulling down the submission data now, will have to spin up another server if I want to do the comment/submission correlation. >>"This dataset will go nicely with the full Reddit Comment Corpus that I released a couple months ago. The link_id from each comment corresponds to the id key in each of the submission…

Is the Comment Corpus still available? I have looked but couldn't see anything obvious (my apologies if I'm being thick here)

Post by tomtoise » Mon, Sep 28, 2015, 11:37 AM UTC