Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is search so bad? A case study

svilentodorov.xyz

Post reply on HN
Print view View original

241–250 of 416 posts

Re: How is search so bad? A case study

#241

Post by CriticalCathed »

DenisM wrote: post #235
erklik wrote: post #224

Earlier quoted context omitted.

> Despite the fact that I've had around 50 interviews and countless articles written about my disclosure, my website zachvorhies.com doesn't show up on Google's search index, even when using the exact url as a query! I am not sure what is happening but I directly searched for your website : zachvorhies.com on Google (in Australia, if that matters), which returned the website as the first result: https://i.imgur.com/Z…

I'm in the US and Google does not display zachvorhies.com when searching for "zachvorhies.com".

Does for me. USA.

[0] https://i.imgur.com/S2rPywz.png

Also, is this guy a Q follower or something? The favicon is a Q.

Re: How is search so bad? A case study

#242

Post by biggestdecision »

MayeulC wrote: post #51
rahulchhabra07 wrote: post #32

I have been thinking about the same problem since a few weeks. The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. Results are not sorted based on relevance or quality but by SEO experts' efforts at making the search results favor themselves. I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surf…

> A new breakthrough heuristic today will look something totally different, just as meritocratic and possibly resistant to gaming. I wonder how much of this could be obtained back by penalizing: 1. The number of javascript dependencies 2. The number of ads on the page, or the depth of the ad network This might start a virtuous circle, but in the end, this is just a game of cat-and-mouse, and website might optimize fo…

Google already penalizes based off payload size/download speed.

Re: How is search so bad? A case study

#243

Post by klingonopera »

reaperducer wrote: post #169
rhacker wrote: post #55

Earlier quoted context omitted.

It is reasonable. It is also likely that whatever meta information reddit is sending back (in headers or tags) is probably not dated correctly for the time of the origin post. Google COULD offer more time machine features and perform diffing on pages. But a reddit "page" will always have content changes, as everything is generated from a database and kept fresh on the page. The ONLY metric therefore Google could use…

It is also likely that whatever meta information reddit is sending back (in headers or tags) is probably not dated correctly for the time of the origin post. That doesn't explain why Google lists the old search results as being from this month, while Duck correctly lists them as being from years past.

Google does cache results, and could via comparison with cache notice changes, and claim, the page was updated sometime between.

I've always wondered, how search engines get a hold of timestamps. Locally with a cached sample, like I explained above, parsing a page's content or some metadata? It's not like the HTTP protocol sends me a "file created/last modified date" along with the payload, does it?

Re: How is search so bad? A case study

#244

Post by dehrmann »

lettergram wrote: post #158

I've been thinking about this for years[1]. The truth is, what Google solved was parsing the search query, not identifying the best results. In fact, Google is not incentivized to give you the best results, they are designed to maximize their revenue, derived from getting you to view / click ads. Google is not a search company, they are an advertising company. The more searches you make, the more revenue they make. T…

I used to work for a company that did product search. Search is hard. The original idea behind Page Rank was really insightful and made search a lot easier, at least until first SEO, then no one linking as much as they used to. The other trick Google figured out was good ML on query results, so all the popular queries have decent results. That sill leave you with search being hard.

Re: How is search so bad? A case study

#245

Post by c3534l »

eyegor wrote: post #238
erklik wrote: post #224

Earlier quoted context omitted.

> Despite the fact that I've had around 50 interviews and countless articles written about my disclosure, my website zachvorhies.com doesn't show up on Google's search index, even when using the exact url as a query! I am not sure what is happening but I directly searched for your website : zachvorhies.com on Google (in Australia, if that matters), which returned the website as the first result: https://i.imgur.com/Z…

[removed, for some reason I thought australia was part of the EU. I'm in the US, also did not see the site link through google]

I'm in the US and I get the same search results. Although if I put in your name, I get your Twitter instead. Not sure why anyone would be searching for a dot com instead of going to it.

Re: How is search so bad? A case study

#246

Post by eyegor »

rahulchhabra07 wrote: post #32

I have been thinking about the same problem since a few weeks. The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. Results are not sorted based on relevance or quality but by SEO experts' efforts at making the search results favor themselves. I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surf…

Here's an xkcd [0] inspired idea. We have several search engines, each with some level of bias. We're not looking to crawl the whole internet because we can't compete with their crawlers. However, we could make a crawler to crawl their results, and re-rank the top N from each engine according to our own metric. Maybe even expose the parameters in an "advanced" search. I'm assuming this would violate some sort of eula though. Any idea if someone has tried this approach?

Edit: thinking more about this post's specific issue, I'm not sure what to do if all the crawlers fail. Could always hook into the search apis for github, reddit, so, wiki, etc. Full shotgun approach.

[0] https://xkcd.com/927/

Re: How is search so bad? A case study

#247

Post by joshuamorton »

zackees wrote: post #219
rahulchhabra07 wrote: post #32

I have been thinking about the same problem since a few weeks. The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. Results are not sorted based on relevance or quality but by SEO experts' efforts at making the search results favor themselves. I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surf…

The real reason why search is so bad is that Google is downranking the internet. I should know - I blew the whistle on the whole censorship regime and walked 950 pages to the DOJ and media outlets. --> zachvorhies.com What did I disclose? That Google was using a project called "Machine Learning Faireness" to rerank the entire internet. Part of this beast has to do with a secret Page Rank score that Google's army of w…

(I work at Google, but don't work on search)

> If wikipedia contains cherry picked slander against a person, topic or website then the raters are instructed to provide a low page rank score

This sounds like a good thing to me. Sites that contain lies, fabrications, and falsehoods should not be as highly ranked as those which do not.

Why should shareholders sure Google for, as far as I can tell from your argument, trying to provide users with a more useful product?

Re: How is search so bad? A case study

#248

Post by CriticalCathed »

Swizec wrote: post #181
AznHisoka wrote: post #167

Earlier quoted context omitted.

“I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surface-level knowledge that I get from competing websites who just want to make money off pageviews.” Is it possible that there is no site providing non fluffy content on your query? For a lot of niche subjects, there really are very few if any substantial content on that topic.

> Is it possible that there is no site providing non fluffy content on your query? For a lot of niche subjects, there really are very few if any substantial content on that topic. “Very few if any substantial” The problem is that google won’t even show me the very few anymore. It’s just fluff upon fluff and depth (or real insight at least) is buried in twitter threads and reddit/hn comments, and github issue discussi…

I mirror your sentiment. It used to be that you could use your google fu and you'd be able to find a dozen relevant forum posts or mail chains in plain text. It's much, much, harder to get the same standard of results. "Pasting stack traces and error messages. Needle in a haystack phrases from an article or book. None of it works anymore."

Yeah, if I know where I'm looking (the sites) then google is useful since I can narrow it to that domain. But if I don't know where to look then I'm SOL.

The serendipity of good results on Google.com is no longer there. And given the talent at google you have to wonder why.

Re: How is search so bad? A case study

#249

Post by ch17z »

eyegor wrote: post #238
erklik wrote: post #224

Earlier quoted context omitted.

> Despite the fact that I've had around 50 interviews and countless articles written about my disclosure, my website zachvorhies.com doesn't show up on Google's search index, even when using the exact url as a query! I am not sure what is happening but I directly searched for your website : zachvorhies.com on Google (in Australia, if that matters), which returned the website as the first result: https://i.imgur.com/Z…

[removed, for some reason I thought australia was part of the EU. I'm in the US, also did not see the site link through google]

Australia isn't in the EU.

For what it's worth, I'm in Australia and have the same search results as erklik.

Re: How is search so bad? A case study

#250

Post by teddyh »

reaperducer wrote: post #185
zozbot234 wrote: post #166

Earlier quoted context omitted.

That's the problem that web directories solve. It's not that you're wrong, it's just largely orthogonal to the problem that you'd need a large crawl of the internets for, i.e. spotting sites about X niche that you wouldn't find even from other directly-related sites, and that are too obscure, new, etc. to be linked in any web directory.

That's the problem that web directories solve Not really. A web directory is a directory of web sites. I can't search a web directory for content within the web sites, which is what a niche search engine would do.

Like WAIS?

https://en.wikipedia.org/wiki/Wide_area_information_server

Post reply on HN