Ask HN: Is 'search' a solved problem? • Hacker News Board

Re: Ask HN: Is 'search' a solved problem?

#51

Post by tallanvor » Wed, Jun 13, 2018, 8:51 AM UTC

nikanj wrote: post #29

If anything, it's an abandoned problem. A lot of companies bought really expensive enterprise search systems, which are sitting dormant because the results are so bad. With advances in spamming, internet/email search is getting to be a harder problem every year. I remember when Google was quite effective in finding what I need, but nowadays it's dismal. As an example, I googled for "storename return policy", and got…

The problem with enterprise search systems is that companies often buy a product expecting it to be as good as a public search engine out of the box, but it doesn't work that way.

Companies can have a great intranet search experience, but it requires time to properly configure the schema, train taxonomies, tune relevance, and understand the resources required to make it work quickly and reliably.

Document processing pipelines often require customization to meet the above requirements, and multi-lingual search may never be perfect within an organization due to differences in the way wordbreaking is done.

Some companies do put a lot of effort into doing this right and getting the best experience possible today, but most don't.

Re: Ask HN: Is 'search' a solved problem?

#52

Post by singularity2001 » Wed, Jun 13, 2018, 9:34 AM UTC

bitL wrote: post #44

majewsky wrote: post #38

Earlier quoted context omitted.

> As an example, I googled for "storename return policy", and got page after page of results. All of them were tagged "missing storename", so just randomly picked return policies from other stores. I think it helps to put the critical search term in quotes. At least in DuckDuckGo, `"storename" return policy` would exclude results that are missing "storename".

DDG seems to have recently switched to the same atrocious imprecise search as Google is running for a few years already... I don't want to fight with the search engine: "you meant this!" -- "nno, find me that" -- "you surely meant this so here it is!" ...

I hate DDG auto DYM (did-you-mean) so much.

Re: Ask HN: Is 'search' a solved problem?

#53

Post by Radim » Wed, Jun 13, 2018, 9:41 AM UTC

halflings wrote: post #42

Radim wrote: post #23

"Search" is too broad to ever be solved. That's like "solving entropy". Google focused on a specific subset — you enter a few keywords or a phrase, and the machine returns the top ~10 links to pre-existing (indexed) web pages. But that's not all there is to search! Challenges: 1. Intranets: internal documents, typically in different modalities (FAQs, support cases, wikis, public pages) and across diverse storages tha…

Definitely valid point (that there are very different types of "search", not just one universal way to do things), but: 2) Google does not do simple keyword matching, and certainly has a strong sense of "semantics". 3) Try searching "Cheap hotels in San Francisco" or "Plumber jobs in Chicago" in Google. Just because it's a single search box does not mean all results are generated/displayed in the same way.

Absolutely. Determining query intent is an ancient, well researched and still active domain. And Google publishes their results regularly (kudos to them).

My point was more that encoding the relevant signals into a single general-purpose search box (a sort-of natural language) is an inherently noisy, ambiguous process. When you know what kind of search you want, it's better to factor out the relevant parameters and feedback loops explicitly, give them a clear UI and search flow. Rather than have users fumble with double quotes and double-guessing the query parser.

Re: Ask HN: Is 'search' a solved problem?

#54

Post by AznHisoka » Wed, Jun 13, 2018, 9:42 AM UTC

obelix_ wrote: post #27

We are still in the stone age when it comes to search. Ask Google who played the mens semifinals of Wimbeldon three years ago and Google will tell you it indexed 6 million pages to provide a link that may or may not have the 4 names I am looking for. Why is it doing all this pointless work? And why is it that dumb in 2018? We have got so used to what it does that lot of people have stopped asking questions about how…

“ask google who played in the semfinals in wimbledon”

You sure about that?https://www.google.com/search?client=safari&hl=en-us&q=who+p...

Re: Ask HN: Is 'search' a solved problem?

#55

Post by kapauldo » Wed, Jun 13, 2018, 11:12 AM UTC

It's definitely not solved from a technical perspective but it's really hard to compete in a business perspective. The market wants good fast search. If you come up with great fast search, it's still hard. The only opportunity I see from the business perspective is to challenge the visual paradigm. Having said that, there are tons of opportunities from the academic perspective such as inferring context, letting users control context, etc.

Re: Ask HN: Is 'search' a solved problem?

#56

Post by Grue3 » Wed, Jun 13, 2018, 12:58 PM UTC

No, it's not solved. Google utterly fails at it.

I'm a person with simple needs. I mostly search in English, sometimes in Russian (which is my native language) and sometimes in Japanese (which I'm learning). These languages use completely disjoint sets of characters so it should be obvious to Google what I'm trying to do (since I'm logged in and they know all about me). There's also a setting that allows me to pick languages that Google should prefer when searching (English is always selected). Now there's a little problem:

1) If I choose any language other than English, Google prefers websites in that language over English even if my query is in English. If I'm searching for programming-related stuff, I don't need it poorly translated into my native language, or any other language! I want the original, on the first page of search results. So any language other than English gets switched off.

2) If Japanese is not switched on, Google thinks that any query that consists only from kanji (a subset of Japanese characters that are also used in Chinese) is in Chinese language, so I get pages and pages of Chinese websites before I see any Japanese ones. Since I can't read Chinese at all, it's completely useless. Now, you might say: of course Google has no way to know if I wanted Japanese, not Chinese. Oh, it knows. I can search for a name of a Japanese person, and Google will display a sidebar with their English Wikipedia article, date of birth and nationality and everything, and yet all the search results will still be in Chinese.

Re: Ask HN: Is 'search' a solved problem?

#57

Post by gwbas1c » Wed, Jun 13, 2018, 1:27 PM UTC

I remember when I used to hear all the time about the semantic web. It sounded more and more like mind reading.

Computers can not read minds.

Re: Ask HN: Is 'search' a solved problem?

#58

Post by dev1n » Wed, Jun 13, 2018, 1:34 PM UTC

Radim wrote: post #23

"Search" is too broad to ever be solved. That's like "solving entropy". Google focused on a specific subset — you enter a few keywords or a phrase, and the machine returns the top ~10 links to pre-existing (indexed) web pages. But that's not all there is to search! Challenges: 1. Intranets: internal documents, typically in different modalities (FAQs, support cases, wikis, public pages) and across diverse storages tha…

Just want to say thank you for gensim [1].

[1]: https://radimrehurek.com/gensim/

Re: Ask HN: Is 'search' a solved problem?

#59

Post by hedora » Wed, Jun 13, 2018, 1:38 PM UTC

Internet search has been solved for about 10 years.

Evidence: Before it sold off to Bing, Yahoo search was quantifiably better than google for a few years (in blind tests where you rip off the branding).

No one cared, because google was good enough.

Having said that, I use duck duck go these days, and occasionally spot check using google. I think the google results have become unusable because they too aggressively map to related concepts, and otherwise second guess what I’ve typed, but there have been endless debates about that on HN, and it’s essentially decided by the user’s taste.

Re: Ask HN: Is 'search' a solved problem?

#60

Post by emiliobumachar » Wed, Jun 13, 2018, 1:41 PM UTC

Grue3 wrote: post #56

No, it's not solved. Google utterly fails at it. I'm a person with simple needs. I mostly search in English, sometimes in Russian (which is my native language) and sometimes in Japanese (which I'm learning). These languages use completely disjoint sets of characters so it should be obvious to Google what I'm trying to do (since I'm logged in and they know all about me). There's also a setting that allows me to pick l…

Seconded. Bilingual support is lacking. For all the profiling they do, they should know my language preferences better than myself.