Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

valohai.com

Post reply on HN
Print view View original

71–80 of 82 posts

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#71

Post by rhizome »

jrumbut wrote: post #66

A very cool demo and I congratulate the author, but I am always a little sad for more data science type demos that try to answer the question (that is proving toxic) "given what I know about you, how can I find a community of people just like you?" I would love to see a subreddit finder that answers questions like "what community would complement your interests?" or "what community needs to hear what you have to say?…

They aren't plug 'n play for advertising, though. "How can I find a community for you," is the charitable flipside to, "here's a community you might like to be a part of," where the community is "Coors Lite purchasers."

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#73

Post by behindsight »

benibela wrote: post #70
behindsight wrote: post #65

Earlier quoted context omitted.

You can also download them from their original mirror at pushshift[1] PS: that BQ you linked is maintained by fhoffa[2] 1: https://files.pushshift.io/reddit/ 2: https://news.ycombinator.com/user?id=fhoffa

Original mirror? But the BQ has 2019_10 while the pushshift has only up 2019-09

Apologies let me clarify: Pushshift has a live api that mirrors reddit which you can use to compile a more recent dataset[0].

The uploader of that BQ has cited Pushshift as their source[1].

PS: In the next couple of days the batched archive data for Q4 of 2019 as well as Q1 of 2020 will be available[2]

0: https://github.com/pushshift/api

1: https://www.reddit.com/r/bigquery/comments/fcyu4m/extended_o...

2: https://www.reddit.com/r/pushshift/comments/fuoe2d/september...

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#74

Post by drusepth »

jedieaston wrote: post #45
whatsmyusername wrote: post #34

Earlier quoted context omitted.

/r/programming is basically "I reposted this from https://news.ycombinator.com"

I've found accounts (possibly bots) that seem to go find the accompanying HN thread, and post a comment from it onto the reddit thread for karma. I sent reddit's anti-evil team a note about it since it's probably karma farming, but they never responded. Maybe it doesn't matter? It's not like we hold exclusive rights to stuff on here, so there's no legal issue, it just seems to be an efficient way of getting karma for…

I used to do something like this any time I detected twin threads (discussing the same URL) across HN and reddit. If anyone asked any unanswered question at one source, one of my bots would ask it at the other sources (plus Quora, usually) and wait for a response elsewhere, then paste that response back to the original OP with a link/citations. Including the citations got me autobanned a few times (which reddit admins graciously removed, repeatedly); if I weren't concerned with plagiarism, bot management would probably have been much smoother.

Would be pretty trivial to skip all the question/answer stuff and just share comments around sites. In a vacuum, I'd say it could be argued that mirroring comments around the Internet would result in good in various ways (sharing information, letting people choose what site they want to use, limiting censorship and/or site downtime, getting answers to people who might not know the best place to ask them, etc).

What you saw was probably karma farming, but could also have been someone trying to help in some abstract way. :)

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#75

Post by duxup »

whatsmyusername wrote: post #34
ChefboyOG wrote: post #20

Earlier quoted context omitted.

HN doesn't get enough credit for the tight rope they walk maintaining this community. I see some people post that HN should expand to other topics a la Reddit, but the team does a great job of maintaining focus. It's not just HN's aesthetic that is minimal and no-nonsense, it's their moderation policies and the tone they set for the community. There is perfect alignment between their approach to content, community, a…

/r/programming is basically "I reposted this from https://news.ycombinator.com"

It would be better if it was just that...

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#76

Post by xigency »

jrumbut wrote: post #66

A very cool demo and I congratulate the author, but I am always a little sad for more data science type demos that try to answer the question (that is proving toxic) "given what I know about you, how can I find a community of people just like you?" I would love to see a subreddit finder that answers questions like "what community would complement your interests?" or "what community needs to hear what you have to say?…

Good point, but I think the goal that you outline is harder and that this type of research is a stepping stone along that path.

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#77

Post by teslademigod1 »

H8crilA wrote: post #68

"What is the penalty for living" -> http://reddit.com/r/Poland , 28% "When should I kill my chicken" -> http://reddit.com/r/csgo , 19% "Am I conscious" -> http://reddit.com/r/INTP , 25% "How to not think" -> http://www.reddit.com/r/howtonotgiveafuck/ , 49% "Is the government evil" -> http://www.reddit.com/r/ENLIGHTENEDCENTRISM/ , 19% "Is the government good" -> http://www.reddit.com/r/CoronavirusUK , 10% "Is the gove…

Poland for the win :D

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#78

Post by cwillu »

rhizome wrote: post #71
jrumbut wrote: post #66

A very cool demo and I congratulate the author, but I am always a little sad for more data science type demos that try to answer the question (that is proving toxic) "given what I know about you, how can I find a community of people just like you?" I would love to see a subreddit finder that answers questions like "what community would complement your interests?" or "what community needs to hear what you have to say?…

They aren't plug 'n play for advertising, though. "How can I find a community for you," is the charitable flipside to, "here's a community you might like to be a part of," where the community is "Coors Lite purchasers."

That's so short sighted though; I'd be so much more likely to engage with an advert that figured out a new thing that would interest me than the usual "you've been reading about sc2 for a week, so have more of the same" nonsense.

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#79

Post by Arimbr »

_v7gu wrote: post #67

Is r/thedonald not included in the database? I'm trying the usual suspect titles, but get nothing.

Yes, unfortunately is not part of the 4k subreddits I trained on. I will retrain it with more subreddits :)

The list of subreddits and an estimation of the performance for each one is on this Google Spreadsheet

https://docs.google.com/spreadsheets/d/1NBY1o85ZiNpcm4tcYhKk...

Re: Show HN: Subreddit Finder - Trained on 4M Reddit Posts from 4K Subreddits

#80

Post by Arimbr »

pmoriarty wrote: post #60

Or you could just ask here: https://old.reddit.com/r/findareddit/

Yes, that is the human solution to the problem. I manually tested a bit on what people ask there :)

On the other hand, the machine is faster and lot of people don't get an answer there or can wait for it. The machine is not necessarily better, just a complement.

Post reply on HN