Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reddit can't build a better search engine

ruky.me

Post reply on HN
Print view View original

231–240 of 289 posts

Re: Reddit can't build a better search engine

#231

Post by jrussbowman »

nivethan wrote: post #226
jrussbowman wrote: post #220

I built https://www.unscatter.com using Reddit to source links for the search index. Last year I added Twitter as another source. I don't think it's a "better" search engine. It's a different lens through which to search. Reddit and Twitter are an information source for what people are talking about. This is why I limit my index to articles that have popped in my Reddit/Twitter input in the last 30 days, deleting any…

Very cool site! Is there a way to get the words in a list instead of a cloud? I am interested in what your thoughts are about being in an echo chamber and getting out of it, but this is a great way to get a high level view of whats happening on reddit.

Right now the terms are available only via the word cloud. I have considered trying to put together an api and also keeping more metrics about each link. For example how many times it's popped up, reddit and twitter users that posted it and which subreddits the post is in. I just haven't gotten around to it.

The concern of it being an echo chamber is one of the major reasons I added Twitter and I still look for more sources. For most of Trump's presidency some form of his name was the top trend 24 hours a day. Crypto is another trend, that while it's great for me because I'm interested in it as well, I question if it's really reflective of what the world is talking about.

I have considered creating some sub-sites as well to try and dig more. Focusing on subreddits for specific categories, but the Twitter api (at least what I can afford which means free) isn't quite as flexible for doing that kind of thing while staying inside my api call limits.

Re: Reddit can't build a better search engine

#232

Post by Karsteski »

14 wrote: post #30
alar44 wrote: post #16

Earlier quoted context omitted.

Their site has been thoroughly trashed already. It's an absolute cesspool of bots and groupthink. When I joined back in 2007, it was a completely different site full of actual discussion and not just cringey self deprecating nerd jokes and memes. It's a shame, but that's how things go. Once a thing gets that big, it gets its soul sucked out.

Avoid r/Canada just a toxic community where mods delete anything that offends them and goes against their beliefs. I honestly think it is also overrun by bots trying to steer an narrative. Everyone just argues and fights now it seems.

I think a lot of the Canada-based subreddits are so political and group-thinky, for lack of a better term, that they are better off avoided at this point. /r/Ontario is the most glaringly obvious example of an echo chamber that I see no point in ever checking it, even though I live in Ontario. My small town's subreddit is decent though, just for the odd news stories every now and then.

Best advice for reddit I can give is to stick to small, focused subreddits. In my case I only visit gaming/tech/dev/podcast related subreddits. The site is only usable like this now, imo.

Re: Reddit can't build a better search engine

#233

Post by pydry »

valar_m wrote: post #230
pydry wrote: post #222

Earlier quoted context omitted.

This used to work well but moderators have "professionalized" and nowadays will viciously stamp out any kind of separarism. Creating a forum has never been hard. Attracting people is. I would imagine this would get worse as money corrupts reddit the same way money corrupted google. Whomever moderates /r/camping probably isnt doing it for free.

> This used to work well but moderators have "professionalized" and nowadays will viciously stamp out any kind of separarism. How would they go about doing that? I see new, competing subreddits being created all the time. Like, what could a mod do to stop me if I wanted to create a new competing sub?

Censor posts even mentioning it in competing subs they moderate?

Maybe i wasnt clear about that.

Even if everybody from an existing subreddit would move to a better one they wont if they dont hear about it.

Re: Reddit can't build a better search engine

#234

Post by gambler »

jrussbowman wrote: post #220

I built https://www.unscatter.com using Reddit to source links for the search index. Last year I added Twitter as another source. I don't think it's a "better" search engine. It's a different lens through which to search. Reddit and Twitter are an information source for what people are talking about. This is why I limit my index to articles that have popped in my Reddit/Twitter input in the last 30 days, deleting any…

> Reddit and Twitter are an information source for what people are talking about.

It's a source of information for what Reddit and Twitter owners want you to see. Both websites are heavily manipulated in a myriad of ways. (The simplest one is via massive amounts of accounts banned for wrongthink.) This is blatantly obvious if instead of passively absorbing news you deep-dive into a specific issue and then look up the discussions and trends, especially on Reddit. Sometimes discussions for major events just aren't there, which is nigh impossible organically on a website with tens of millions of users.

Re: Reddit can't build a better search engine

#235

Post by jrussbowman »

gambler wrote: post #234
jrussbowman wrote: post #220

I built https://www.unscatter.com using Reddit to source links for the search index. Last year I added Twitter as another source. I don't think it's a "better" search engine. It's a different lens through which to search. Reddit and Twitter are an information source for what people are talking about. This is why I limit my index to articles that have popped in my Reddit/Twitter input in the last 30 days, deleting any…

> Reddit and Twitter are an information source for what people are talking about. It's a source of information for what Reddit and Twitter owners want you to see. Both websites are heavily manipulated in a myriad of ways. (The simplest one is via massive amounts of accounts banned for wrongthink.) This is blatantly obvious if instead of passively absorbing news you deep-dive into a specific issue and then look up the…

Manipulated or not, I do agree it's an echo chamber. I added Twitter in attempt of breaking the echo chamber aspect but wasn't as successful as I hoped. I do occasionally look for other sources to add but finding similar sites that can match the volume of those two is difficult. Which makes it more difficult to figure out how to weigh other sites results to those two.

This discussion did make me poke around some more. I may consider using the free tier of Bing's api just to pull in trending topics.

Re: Reddit can't build a better search engine

#236

Post by ksec »

quixoticaxolotl wrote: post #206

I've searched for Reddit in the past because Reddit is the only site where I can get community answers to an important question. In those cases, the community aspect mattered to me - for example, I recently moved to Boston, so I solicited Bostonian's recommendations from r/boston; my partner and I were having troubles, and I wanted to hear from people who'd gone through similar experiences; I wanted to learn about co…

> the community aspect mattered to me

That is why Google was worried about Social Media / Facebook eating them, and hence the creation of Google+. When most of the important, or valuable answer are on the social network, search engine will cease to be the entry point of internet. And in many countries, Facebook and Whatsapp really is "the" Internet.

And after that there seems to be a gentleman agreement, Zuckerberg wont enter search. And Google wont enter Social.

And actually I use HN for searches as well. From Product recommendation to many other niche subject.

Re: Reddit can't build a better search engine

#237

Post by Traster »

zozbot234 wrote: post #161
Traster wrote: post #159

I completely disagree with some of the points here. There seems to be this weird assumption from people that Reddit is some organic source of information that hasn't been polluted by advertisers and SEO. But the truth is that it absolutely has, the difference is that the way you game the system is different. Google is this crazy algorithm and no one really knows how it works, so people just throw shit at it. Keyword…

> What you do get though, is highly unreliable information about anything that could be monetized. So it's just like the rest of the Internet. And at least in Reddit (or, more to the point, the newer federated lookalikes - Reddit is dying and losing its highest value and most committed users) you can go look for the tiny niche communities focused around that interest, where users will be better focused on policing in…

The problem with reddit isn't different to the problem of astroturf in other places, it's that it's a highly concentrated form of it since it's so popular it's a highly valuable target.
Post reply on HN