Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is search so bad? A case study

svilentodorov.xyz

Post reply on HN
Print view View original

51–60 of 416 posts

Re: How is search so bad? A case study

#51

Post by MayeulC »

rahulchhabra07 wrote: post #32

I have been thinking about the same problem since a few weeks. The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. Results are not sorted based on relevance or quality but by SEO experts' efforts at making the search results favor themselves. I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surf…

> A new breakthrough heuristic today will look something totally different, just as meritocratic and possibly resistant to gaming.

I wonder how much of this could be obtained back by penalizing:

1. The number of javascript dependencies 2. The number of ads on the page, or the depth of the ad network

This might start a virtuous circle, but in the end, this is just a game of cat-and-mouse, and website might optimize for this as well.

What we might need to break this is a variety of search engines that uses different criteria to rank pages. I suspect it would be pretty hard, if not impossible, to optimize for all of them.

And in any case, frequently change the ranking algorithms to combat over-optimization by the websites (as that's classically done against ossification for protocols, or any overfitting to outside forces in a competitive system).

Re: How is search so bad? A case study

#53

Post by LastZactionHero »

You're located in Berlin. It found a page on Reddit about buying phones in Berlin.

Not saying you're wrong about the dates but... I dunno... seems like an odd query. "Phone to buy?" And why not just search Reddit?

This site feels like we're just complaining about nothing these days. (Downvote away!)

Re: How is search so bad? A case study

#54

Post by bsanr2 »

yoz-y wrote: post #36
echelon wrote: post #23

Earlier quoted context omitted.

Imagine if instead of kneecapping XHTML and the semantic web properties it had baked in, Google had not entered into the web browser space. We might be able to mark articles up with ` `, and set their subject tags to the URN of the people, places, and things involved. We could give things a published and revised date with change logs. Mark up questions, solutions, code and language metadata. All of that is extremely…

I work for Google but not on search. I think in this case semantic web would not work, unless there was some way to weed out spam. There are currently multiple competing microdata formats out there than enable you to specify any kind of metadata but they still won't help if spammers fill those too. Maybe some sort of webring of trust where trusted people can endorse other sites and the chain breaks if somebody is fou…

In the interests of not causing a crisis when Top Level Trust Domain endorses the wrong site and the algorithm goes, "Uh uh," (or the endorsement is falsely labeled spam by malicious actors, or whatever), maybe the effect decreases the closer you are to that top level.

But that's hierarchical in a very un-web-y way... Hm.

Re: How is search so bad? A case study

#55

Post by rhacker »

prostheticvamp wrote: post #39
ma2rten wrote: post #11

This article is essentially just complaining that DDG and Google don't have special parsing for reddit pages ("How come it doesn't know that thread didn't get many upvotes?", "How come it thinks some change to the site's layout was an update to the page?") Maybe if you want to search reddit, the best search engine is the search bar on reddit.com.

But those pretend complaints aren't his complaints. His complaint is "why does this archived reddit page from six years ago without any updates come up on search results for 'things within the past month'?" Which is... reasonable.

It is reasonable. It is also likely that whatever meta information reddit is sending back (in headers or tags) is probably not dated correctly for the time of the origin post.

Google COULD offer more time machine features and perform diffing on pages. But a reddit "page" will always have content changes, as everything is generated from a database and kept fresh on the page. The ONLY metric therefore Google could use would be whatever meta tag or header tag that reddit provides.

Re: How is search so bad? A case study

#58

Post by klint »

basscomm wrote: post #44
tyingq wrote: post #7

Earlier quoted context omitted.

My guess is that suppressing spammy pages got too hard. So they applied some kind of big hammer that has a high false positive rate. You're getting the best of what's left. Maybe also some quality decline in their gradual shift to less hand weighted attributes and more ML.

My guess is that Google et al are all hell-bent on not telling you that your search returned zero results. They seem to go to great lengths to make sure that your results page has something on it by any means necessary, including: searching for synonyms for words I searched for instead of the specific words I chose, excluding words to increase the number of results (even though the words they exclude are usually the…

I was going to reply with something along the same lines. Dropping the keyest keywords is a particularly big pet peeve of mine.

For those "needle in a haystack" type queries, instead of pages that include both $keyword1 and $keyword2, I often get a mix of the top results for each keyword. The problem is compounded by news sites that include links to other recent stories in their sidebars. So I might find articles about $keyword1 that just happen to have completely unrelated but recent articles about $keyword2 in the sidebar.

It also appears that Google and DDG both often ignore "advanced" options like putting exact phrase searches in quotation marks, using a - sign to exclude keywords, etc.

None of this seems to have cut down on SEO spam results either, especially once you get past the first page or two of results.

I suspect it all comes down to trying to handle the most common types of queries. Indeed, if I'm searching for something uncomplicated, like the name of the CEO of a company or something like that, the results come out just fine. Longtail searches probably aren't much of a priority, especially when there's not much competition.

Re: How is search so bad? A case study

#59

Post by JohnJamesRambo »

vanderZwan wrote: post #46
JohnJamesRambo wrote: post #35

Earlier quoted context omitted.

Goodhart‘s Law in action. I wonder how we make a new measure that buys us more time? https://en.wikipedia.org/wiki/Goodhart's_law

The backtick in your link broke it: https://en.wikipedia.org/wiki/Goodhart's_law (Were you on mobile and using a smart keyboard?)

I manually added the comma on a mobile smart keyboard. :) Didn't know that doesn't work haha.
Post reply on HN