Reddit can't build a better search engine • Hacker News Board

Re: Reddit can't build a better search engine

#101

Post by kumarvvr » Thu, Feb 17, 2022, 8:36 AM UTC

ddtaylor wrote: post #28

> However, once Reddit creates a search engine, and once people get to know that there is an opportunity to game the system and create a financial opportunity, people will abuse that system and we will be back to the place where we are now. SEO stuffed websites. I help run a Reddit website and open source project for managing scheduled posts [1] and we see a lot of garbage there, sadly. Over the years the site has pr…

This is already happening. Advertisers create normal looking fake accounts, and subtly inject product placements into discussions.

The best way to market a thing is to not make people realize they are being marketed with.

I remember a few years ago, there was a post (AMA) by someone who does this and his whole job was to maintain hundreds of normal looking accounts and post SEO friendly stuff, good words about products, etc on reddit.

Re: Reddit can't build a better search engine

#102

Post by buro9 » Thu, Feb 17, 2022, 8:37 AM UTC

There is already a great search engine, it is Google.

The problem is that it contains an overwhelming volume of spam and auto-generated sites.

To solve that, we all use the best search engine and apply site filters. Sites where we know other humans may have already discussed it.

We filter for forums - and Reddit is the largest of those - and we filter for specific websites we already trust.

Sometimes I filter `site:news.ycombinator.com companyname`, sometimes it's `site:dcrainmaker.com best gps watch`, sometimes it's `site:lfgss.com bottom bracket cable guide de rosa`.

But if I don't know about a specialist website, then it's `site:reddit.com searchterm`.

The search engine exists, it's just full of garbage with no effective way to search minus the garbage.

I'd take another DMOZ, a human curation of specialist sites. Then I'd have something better than Reddit too... I'd have the right filter to make every search useful.

Re: Reddit can't build a better search engine

#103

Post by stefs » Thu, Feb 17, 2022, 8:38 AM UTC

i remember history very differently. the old search engines before google analyzed only page content in isolation, which lead to SEO sites appending invisible blocks of keywords at the bottom or putting loads of barely related keywords in the page title.

google was the one that managed to break this cycle with pagerank. suddenly, the spamword filled trash sites were worthless because nobody (of worth) linked to them. on the other hand blogs flourished, because their structure worked well with pagerank.

it took a long time for the spammers to re-build their networks and strategies to adapt to the situation, but even then having a well-written and regularly updated company weblog usually did more for your search ranking than other SEO trickery.

Re: Reddit can't build a better search engine

#104

Post by lordnacho » Thu, Feb 17, 2022, 8:39 AM UTC

pojzon wrote: post #92

conjecTech wrote: post #34

This thesis is off. It's not as if Reddit just appeared on the internet yesterday. It's been a place people look for this kind of info for a long time, and people have been trying to game it for just as long. And it already has a good search engine - that just happens to be Google. So it still leaves the question: why does Reddit still have such a high signal-to-noise ratio. A few things I've noticed: 1. A lot of con…

Recently have noticed a business creating fictional threads on reddit (multiple on different communities) working as an AD to sell something. Example: „What is the cheapest best cooling for cpu now?” All discussions point to product that more experienced users know its crap. But still the wrong product is at the top with most (botted) upvotes. Companies realized that ppl look for opinions on reddit -> that created de…

Yeah initially I thought Reddit was useful for "give an opinion" type queries, especially where you're getting answers to a newly posted query. Eg my brother managed to get a specialist doctor to give an opinion about our mom's treatment.

But for the vast majority of commercially relevant searches, it will get sunk quite soon. It can't be terribly hard to hook up a GPT style bot to harvest karma on various accounts and then use that to gain credibility on "which headphones should I buy" type queries. You might not even need to automate it, a farm of real people could potentially be useful for that, I bet that's offered by someone out there already.

Re: Reddit can't build a better search engine

#105

Post by karatinversion » Thu, Feb 17, 2022, 8:41 AM UTC

lrem wrote: post #98

pojzon wrote: post #92

Earlier quoted context omitted.

Recently have noticed a business creating fictional threads on reddit (multiple on different communities) working as an AD to sell something. Example: „What is the cheapest best cooling for cpu now?” All discussions point to product that more experienced users know its crap. But still the wrong product is at the top with most (botted) upvotes. Companies realized that ppl look for opinions on reddit -> that created de…

Closed communities is where it's at... But they have the obvious inherent problem: they don't scale. If a closed community lets in too many people, it becomes a somewhat obscured open community, with all the downsides. And if it doesn't, well, you are unlikely to get in on a topic you are interested in.

One of the lessons of the last decade (?) is that if a community scales, subverting it also scales

Re: Reddit can't build a better search engine

#106

Post by spiderfarmer » Thu, Feb 17, 2022, 8:43 AM UTC

I sometimes wonder if we would have had very good search engines if one of the Tech Giants invested in a huge amount of manual information entry/organization instead of optimising around spam and junk websites.

Like Encyclopedia Brittanica, but on steroids. With 500.000 editors and a couple millions volunteers instead of a couple hundred editors.

Google has 135.000 people on staff right now, all very highly paid. They could have a massive advantage if they had their own unique dataset.

Re: Reddit can't build a better search engine

#107

Post by yetanother-1 » Thu, Feb 17, 2022, 8:43 AM UTC

buro9 wrote: post #102

There is already a great search engine, it is Google. The problem is that it contains an overwhelming volume of spam and auto-generated sites. To solve that, we all use the best search engine and apply site filters. Sites where we know other humans may have already discussed it. We filter for forums - and Reddit is the largest of those - and we filter for specific websites we already trust. Sometimes I filter `site:n…

How would you search private forums (subreddits) that you are part of, but not accessible for Google?

Re: Reddit can't build a better search engine

#108

Post by betwixthewires » Thu, Feb 17, 2022, 8:46 AM UTC

nemothekid wrote: post #27

14 wrote: post #18

Earlier quoted context omitted.

No. There still are forums on lots of topics. But search how to fix X and instead of prioritizing a forum site with answers you get 5 top results trying to sell you the parts to fix X. The answers are always way down the list.

> There still are forums on lots of topics. What is Reddit if not the largest forum in the world? You have the same gripe; you are looking for a web where people share things for free (like in forum); and when Google first existed most of the web is like that. And if google were to prioritize forums; then SEO blackhats would do everything in their power to make every website look like a forum; and then you are back t…

Explain away how literally verbatim searching with quotes doesn't return verbatim results. Or how searches that match the very title of an article returns mountains of SEO spam before listing the article, if it shows up at all.

If there's an algorithm being gamed that causes that, that algorithm was designed explicitly to be gamed.

Re: Reddit can't build a better search engine

#109

Post by betwixthewires » Thu, Feb 17, 2022, 8:50 AM UTC

zemo wrote: post #31

> Reddit posts are good because the people who create these posts or make comments are doing it to share their knowledge. people post to Reddit (and here) not to share their knowledge but because they get a little squirt of dopamine from upvotes and downvotes and replies.

I find that most people on this site use it to engage in discussion, to learn and to teach if they think they have something of value to say. For the most part, you're right about Reddit these days. When I used that site I'd check votes every time I was done scrolling. This site I check threads to see if anyone responded to me with anything I'd like to reply to.

Re: Reddit can't build a better search engine

#110

Post by buro9 » Thu, Feb 17, 2022, 8:52 AM UTC

yetanother-1 wrote: post #107

buro9 wrote: post #102

There is already a great search engine, it is Google. The problem is that it contains an overwhelming volume of spam and auto-generated sites. To solve that, we all use the best search engine and apply site filters. Sites where we know other humans may have already discussed it. We filter for forums - and Reddit is the largest of those - and we filter for specific websites we already trust. Sometimes I filter `site:n…

How would you search private forums (subreddits) that you are part of, but not accessible for Google?

I'm already not signed in to Reddit so this already applies. It turns out most forums default to public as that drives organic growth that sustains them over time. There are enough public fora to not have to seek out private ones to answer things.