Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is search so bad? A case study

svilentodorov.xyz

Post reply on HN
Print view View original

91–100 of 416 posts

Re: How is search so bad? A case study

#91

Post by zozbot234 »

jillesvangurp wrote: post #72

It's a hard problem because what is relevant is inherently subjective and context specific and only a minority of users uses the advanced search functionality so it is also not a big priority to solve it. Both Google and Duck Duck Go optimize for the simple use case where there's a bit of user context and some short query that the user typed. That's what needs to work well. For that Google is still pretty good. I try…

Hacker News doesn't even have search at the moment. It just redirects you to some crappy external site.

Re: How is search so bad? A case study

#92

Post by tsian2 »

yusef555 wrote: post #41
aldoushuxley001 wrote: post #12

Earlier quoted context omitted.

everyone knows the best way to search reddit is via google. The search engine bar in reddit is for optics only.

But does anyone know why search on Reddit is broken? Perhaps intentionally? I don't want to get tin foil hatty but perhaps more not readily apparent false positives = more user clicks = more revenue via ad serving?

I often wonder why some fairly large companies that rely heavily on their own website don't seem to put more than a sole web developer worth of resources into them. Reddit fits into that category for me (Reddit has 400 employees).

Initially I had the impression that search was hard to implement. However, spending a work week figuring it out with ElasticSearch, Solr and Sphinx changed my mind. Getting the solution to work with the scale of the website would take more work, but all the know-how is there, and they could put a whole team to the task for a month.

Re: How is search so bad? A case study

#93

Post by zozbot234 »

allovernow wrote: post #40

I've been screaming about this for years and only recently have people begun agreeing with me - and I know exactly what the major problems are, and they are synergistic: 1. SEO has totally warped result rankings. Now instead of getting results which naturally match my keywords because of content, I'm presented with almost exclusively commercial websites which are trying to sell me something. Gone are the days where y…

Well, '"roof repair"' was never a good search for obvious reasons. But you'd expect a search like '"how to" repair roof' to mostly filter out sites providing roof-repair services - and if a search engine doesn't do that properly (because it ignores the "how to" part as irrelevant even though it's in quotes!) that's just broken.

Re: How is search so bad? A case study

#94

Post by zozbot234 »

derefr wrote: post #64
MayeulC wrote: post #51

Earlier quoted context omitted.

> A new breakthrough heuristic today will look something totally different, just as meritocratic and possibly resistant to gaming. I wonder how much of this could be obtained back by penalizing: 1. The number of javascript dependencies 2. The number of ads on the page, or the depth of the ad network This might start a virtuous circle, but in the end, this is just a game of cat-and-mouse, and website might optimize fo…

You could even have all this under one roof: one common search spider that feeds this ensemble of different ranking algorithms to produce a set of indices, and then a search engine front end that round-robins queries out between the different indices. (Don’t like your query? Spin the algorithm wheel! “I’m Feeling Lucky” indeed .)

The Common Crawl is a thing already. Unfortunately, a "full" text crawl of the internets is a YUUUGE amount of data to manage, and I can't think of anything that could change that in the foreseeable future. That's why I think providing a federated Web directory standard, ala ODP/DMOZ except not limited to a single source, would be a far more impactful development.

Re: How is search so bad? A case study

#95

Post by thrwaway69 »

OJFord wrote: post #33
thrwaway69 wrote: post #28

Earlier quoted context omitted.

Given how relevant forums and discussion spaces are. One would think there would be some standardized structure for it so you can search for comments or posts with criteria all over the internet.

There are 'standards' (and an XKCD comic) for that. See schema.org for example. Google does use it, last time I used it there was even tooling for it in the 'check how my site is to crawl' console, whatever it's called.

https://schema.org/upvoteCount

Yet I haven't seen even one instance of this anywhere. :(

Re: How is search so bad? A case study

#96

Post by zozbot234 »

aaron-santos wrote: post #69
MayeulC wrote: post #51

Earlier quoted context omitted.

> A new breakthrough heuristic today will look something totally different, just as meritocratic and possibly resistant to gaming. I wonder how much of this could be obtained back by penalizing: 1. The number of javascript dependencies 2. The number of ads on the page, or the depth of the ad network This might start a virtuous circle, but in the end, this is just a game of cat-and-mouse, and website might optimize fo…

Merging js deps into one big resource isn't difficult. The number of ads point is interesting though. How would one determine what is an ad and what is an image? I have my ideas, but optimizing on this boundary sounds like it would lead to weird outcomes.

Adblockers have to solve that problem already. And it's actually really easy because "ads" aren't just ads unfortunately, they're also third-party code that's trying to track you as you browse the site. So it's reasonably easy to spot them and filter them out.

Re: How is search so bad? A case study

#97

Post by klingonopera »

zozbot234 wrote: post #85
JohnJamesRambo wrote: post #59

Earlier quoted context omitted.

I manually added the comma on a mobile smart keyboard. :) Didn't know that doesn't work haha.

That damn ‘Smart Quotes’ misfeature is still causing havoc even after 30 years.

Nitpick: It's actually an apostrophe " ' ", not a backtick/grave accent " ` " or comma " , " :D

https://en.wikipedia.org/wiki/Apostrophe

https://en.wikipedia.org/wiki/Grave_accent#Use_in_programmin...

https://en.wikipedia.org/wiki/Comma

Re: How is search so bad? A case study

#98

Post by scottlocklin »

I agree google is shit, which is why I never use it. They're too busy being woke to run a search engine which compares to what they were doing 10-15 years ago. It's pretty obviously entirely that; smart people don't want to work in dying Brezhnevian bureaucratic hellscapes.

Try Yandex for an example of a much smaller company doing a fine job at search:

https://yandex.com/search/?text=reddit%20phone%20to%20buy&lr...

Produces exactly what OP was looking for. QED.

Qwant isn't bad either; don't remember if they piggyback off of other search results:

https://www.qwant.com/?q=reddit%20phone%20to%20buy&t=web

Re: How is search so bad? A case study

#99

Post by Grimm1 »

This is something I'm currently focused on solving actually. Currently on Show a ways down is the alpha but the basic gist is a search engine focused on discovery while maintaining topical relevance. We thoroughly agree that search results are kind of garbage on major engines at the moment.

Re: How is search so bad? A case study

#100

Post by thom »

cgriswald wrote: post #68
baggy_trough wrote: post #15

Earlier quoted context omitted.

I've been using DDG as a good enough search engine for most things, but when I sometimes fall back to Google, it blows me away how many ads are on the page pretending to be results!

'if not ddg(search) { ddg("!g " + search) }' has been my go-to method for awhile now; but as time has progressed, the results from DuckDuckGo have either been getting better, or the Google results have been getting worse; because usually if I can't find it on DDG now, I can't find it on Google either.

I use DDG by default, but I can feel myself mentally flinching unless I basically know what I'm looking for already (i.e. I know I'll end up on StackOverflow). When I'm actually _searching_, it's useless, and I'll always !g.
Post reply on HN