Are you breaking Reddit TOS by storing/hosting posts? If someone deletes their post on Reddit it will still be stored and available on your site.
Why would OP care about reddit TOS?
An ImageNet-like text classification task based on Reddit posts
11–16 of 16 posts
Re: An ImageNet-like text classification task based on Reddit posts
#12You can get similar raw data using these public bigquery datasets: https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit...
Re: An ImageNet-like text classification task based on Reddit posts
#13Re: An ImageNet-like text classification task based on Reddit posts
#14Are you breaking Reddit TOS by storing/hosting posts? If someone deletes their post on Reddit it will still be stored and available on your site.
I'm not too worried about this. The number of publicly available datasets of reddit posts , many of them hosted on kaggle or bigquery (both owned by google) is very large, which suggests to me that reddit don't mind about this in the same way that say, Twitter does. It was also a deliberate decision to not include the reddit usernames in this data, and I personally don't think this would be a great resource to try and break someones privacy, compared to what else is out there.
That said, if either Kaggle or reddit do have a problem with this I won't hesitate to remove it.
Re: An ImageNet-like text classification task based on Reddit posts
#15The post mentions not getting great results with OpenAI Transformer. I haven't tried that, but using a similar framework, ULM-FiT, I narrowly beat the fasttext benchmark on a 250-class dataset we use internally. I will follow up with how it does on this data set.
I'd be very interested to know, thanks!
Re: An ImageNet-like text classification task based on Reddit posts
#16This looks fantastic. In particular, the focus on many-class classification is important, it's a common real-world task that is often overlooked. I have some suggestions: More types of baseline accuracy measure would be useful, eg. accuracy, and micro and macro f1 with unbalanced classes. It would be very useful to know inter-annotator agreement for the manual classification and human performance for the task of iden…
A nice comment, good to see other people are thinking about this! I agree with you about the imbalanced classes, I do have a copy of this, the main issue is that the way this dataset was created was only looking at subreddits which include 1000 posts or more, meaning that class imbalance is somewhat unrealistic, if I do publish an imbalanced version it will include all subreddits, not just the carefully selected 1013.
re: the reason for the artifice. First of all note that none of the labels in here are exactly superficial - I have made a taxonomy - but I have only used this to filter out subreddits - I did not combine the posts from different subreddits in the same category here. The main reason was to combat the fact that these are not great labels otherwise - many subreddits are subsets of others - e.g. you have r/gaming -> r/finalfantasy -> r/FFVIII - and you don't know that this follows a hierarchy a priori (N.B. categorising all subreddits would require significant resources).
Worse than this, you have subreddits that don't really follow any obvious kind of categorisation, e.g. r/askreddit (by far the most populous in terms of self-posts), or more randomly, subreddits devoted to podcasts like r/joerogan - they are basically places where people go for broad, likeminded chat, and they can overlap with just about anything. I would argue that this is actually not always realistic - for the examples I have worked on in the past, labels were reasonably unambiguous.