Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Is 'search' a solved problem?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 125 posts

Re: Ask HN: Is 'search' a solved problem?

#101

Post by severine »

ma2rten wrote: post #62
nikanj wrote: post #46

Earlier quoted context omitted.

This being relatively large chain store, I'm quite certain there should be forum posts or reddit discussions about their return policy. I have no way of proving they exist, seeing as how Google is shit at finding stuff. "So what's the problem here"? The problem is, the search engine threw away one third of my query terms, and then gave me all chaff. Apparently I should've thrown in some random punctuation to convince…

Can you just tell the name of the store? I want to see for example what happens if I select verbatim mode.

Example with Target, in DDG, verbatim: https://duckduckgo.com/?q=%22target%22+%22return+policy%22&t...

Seems to work.

Re: Ask HN: Is 'search' a solved problem?

#102

Post by severine »

andy_ppp wrote: post #66

I can give a list of things I'd like in a search engine: 1) Sticky topics - if I'm at work and I type Flow I am way more likely to mean Facebook's js library than I am Flow energy. 2) Different views on the information (Grid view/masonary that makes sense) 3) Ability to search within a set of search results 4) Ability to customise the algorithm with other programs I can write/plugins 5) Many more things for programme…

Great list, +1 to everything, +10 to #1, #3.

Re: Ask HN: Is 'search' a solved problem?

#103

Post by stghndthrowaway »

I've always held that the quality of Google services is directly proportional to what Google engineers need for their jobs. Paste a stack trace in the search box? It's uncanny how it will take me straight to the one blog post that tells me what my problem is, and the best solution for it (even if it doesn't contain that stack trace!). Recently I've been working in the theatre, and it's shocking just how bad Google is at searching for theatrical terms.

Part of the problem is that many are generic English words that have special meaning on stage -- but when I add a word like "theatre" to try to clarify, Google assumes I want to buy tickets to a musical. Adding other terms doesn't help much, since there are so many indexed webpages which just happen to have all of those words. It doesn't help, either, that lots of terms have many synonyms (and alternate spellings) which are also common English words. I spent 20 minutes the other day trying every combination of related words and synonyms before Google found a page with the information I needed. Good luck trying to find information on how to make a "return" (the stage object) in Google!

Try a Google Images search for "clew", and compare a search for "clew theatre". Then try to figure out how many of the latter images are actually hits for a silently-autocorrected "crew". (Yes, I can use - and "" to help control this, but why do I need to quote every word I type in Google?) They say their goal is organizing the world's information, but it really looks like they care more about helping the users who want to buy cheap things online.

Theatre is in no way unique here. Most fields I've touched that don't have SEO professionals working to sell products online have terrible presence in Google. At this point, I believe Google was the first competent search engine, and now they're just using that position to make people who want to be found figure out how to get good Google juice.

Is search "solved" if it only works when we have professionals whose job it is to ensure their content is searchable? It's a hard problem, and maybe what we've got is about as good as it gets for the (commercial) web of today, and we need a new system that's naturally more amenable to search. The "semantic web" didn't really pan out, as it was pretty heavy, but why can't someone just slap a meta tag that says dewey="792" on their pages and then have Google let me search by that?

Search as a competitive business and method to search the unstructured web is done. Search as a means to find information is still wide open.

Re: Ask HN: Is 'search' a solved problem?

#104

Post by taurine »

DARPA/IARPA now invests in search 2.0: Instead of asking it to retrieve stored information, you can instruct an agent/search bot to perform tasks for you.

For instance, one should be able to search for: "who is the leader of this IRC hacker group?" "where can heroine be bought on the deep web?" "Is there women trafficking going on behind that log-in wall?" and then an intelligent agent is dispatched, avoiding/crossing roadblocks, like log-in forms, and will eventually bring you the answer.

Coursera has more on the basics, like: https://www.coursera.org/learn/text-retrieval

OpenAI has more on the current happenings of creating more intelligent search bots: https://github.com/openai/universe

Other possible future research areas in information retrieval include being able to search for services ("Where is cheapest taxi service for current location?") and an integration with IOT.

Re: Ask HN: Is 'search' a solved problem?

#105

Post by Jasper_ »

maaark wrote: post #82
tomsmeding wrote: post #77

Earlier quoted context omitted.

I think enclosing the word you really want in "quotes" forces Google to not discard that term.

I thought you were supposed to do "+ImportantWord LessImportantWord". Neither seems entirely reliable.

The +ImportantWord syntax was recycled for Google Plus searches at some point and replaced with "ImportantWord". And now to get a "Important Set of Words" query to work, you now have to enter some special "Verbatim" mode by going into Search Tools and clicking on a special checkbox.

Re: Ask HN: Is 'search' a solved problem?

#106

Post by chriswarbo »

One aspect that I don't see mentioned is that APIs for search have basically disappeared from the major players, likely due to them being expensive (programs can hammer an API faster than a human with a text box) and lacking a revenue model (no human may be involved, making advertising useless).

This has caused "search" to devolve into "human using Web browser types natural language in box, human is presented with results (some semantic, most just links to things other people wrote, some advertisments around the side), human reads through results to see if any are useful to them".

This is certainly useful, and I rely it all the time, but it's not the pinnacle of what search could be. It's like if `grep` didn't pipe to stdout, but instead popped up an alert box for each line like "Your query '.*' matched the line 'foo'. Try the new McDonalds saver menu today! [Next/Cancel]", it would still be useful, but nowhere near as useful as piping to stdout.

Many years ago Google provided an API to their search engines, which applications could build on to be more "smart". This could have paved the way to much better software: for example, imagine a prolog system where all of Google's knowledge could be used by the calculations.

That path was mostly abandoned since there's no scalable incentive to make it operate, much like the semantic Web. Rather than opening up databases to empower others, it's much more profitable to keep them walled off behind a few limited, pay-per-use interfaces (e.g. "paying" by showing a human some adverts). Attempts to bypass or abstract over these interfaces are hit with rate limiters, Recaptcha checks, etc.

Re: Ask HN: Is 'search' a solved problem?

#107

Post by Balgair »

arthev wrote: post #67

Given that google is the de facto standard ATM and often returns irrelevant results, I'd wager that search isn't solved. "That does not by itself mean there's room for a new search engine, but lately when using Google search I've found myself nostalgic for the old days, when Google was true to its own slightly aspy self. Google used to give me a page of the right answers, fast, with no clutter. Now the results seem i…

Is it the MBAs that do this to good products? Google maps was better 7 years ago too.

The best way I have heard this process described is 'the elves leaving middle earth' : https://steveblank.com/2009/12/21/the-elves-leave-middle-ear...

Anyone else know why Google is going full Kodak?

EDIT: It's the SEO/spammers/viruses type folks, isn't it? Seems like a good candidate.

Re: Ask HN: Is 'search' a solved problem?

#108

Post by blihp »

In 1994 there were people who believed search was 'solved' thanks to Yahoo! providing a comprehensive index of the Web (which actually was doable for a couple of years) Then the Internet exploded in size so in 1996 there were people who believed search was 'solved' thanks to AltaVista. The Internet continued to grow and thanks to early SEO techniques like keyword stuffing, an opening existed for Google to fill when they 'solved' search with 'The Algorithm.' Now we're in the midst of search being 'solved' again via ML/DNN's. And I'm sure it will be 'solved' again and again in years to come. So to answer your question about if it's solved, you'd first have to specify which time? :-) Search is a very, very large space and will likely not be solved anytime soon.

Also, don't confuse the business/tech press definition of solved (i.e. a dominant player raking in large piles of money and shutting out competition) with said problem actually being solved as in there does not exist a better way to attack the problem. Granted, when the business/tech world considers a problem solved, this often just means that the lions share of (known) financial incentives are no longer 'low-hanging fruit'... until a challenger figures out a way to blow up the incumbents business model.

Re: Ask HN: Is 'search' a solved problem?

#109

Post by basch »

slyfocks wrote: post #36
nikanj wrote: post #29

If anything, it's an abandoned problem. A lot of companies bought really expensive enterprise search systems, which are sitting dormant because the results are so bad. With advances in spamming, internet/email search is getting to be a harder problem every year. I remember when Google was quite effective in finding what I need, but nowadays it's dismal. As an example, I googled for "storename return policy", and got…

Lately, it has felt like Google has become more optimized for less intelligent/tech-savvy users (i.e. people who search using natural language e.g. “tell me the store policy for target please”). In these cases many of the words used are either irrelevant or counterproductive. I’ve noticed Facebook’s search has started catering to the lowest common denominator as well. Searching for (made-up example) “Louis Potter” us…

facebook used to let you do direct knowledge graph searches

"restaurants liked by people who like Joe's Pancakes and Green Dragon Mexican Pizza"

"movies liked by people who are friends with people who like Back to the Future and Arrested Development"

"people who live nearby and like Green Dragon Mexican Pizza and Arrested Development"

"friends of friends of friends who like Arrested Development and Joe's Pancakes"

"Friends of Megan Albert friends who are women named "Erin" Chipotle employ"

I think you can see why they removed it.

this URL might lead to something like "restaurants liked by people who like Snow White" https://www.facebook.com/search/130104022710/likers/pages-li...

Re: Ask HN: Is 'search' a solved problem?

#110

Post by bobosha »

Radim wrote: post #53
halflings wrote: post #42

Earlier quoted context omitted.

Definitely valid point (that there are very different types of "search", not just one universal way to do things), but: 2) Google does not do simple keyword matching, and certainly has a strong sense of "semantics". 3) Try searching "Cheap hotels in San Francisco" or "Plumber jobs in Chicago" in Google. Just because it's a single search box does not mean all results are generated/displayed in the same way.

Absolutely. Determining query intent is an ancient, well researched and still active domain. And Google publishes their results regularly (kudos to them). My point was more that encoding the relevant signals into a single general-purpose search box (a sort-of natural language) is an inherently noisy, ambiguous process. When you know what kind of search you want, it's better to factor out the relevant parameters and f…

@Radim - is this something you are doing with your current startup? p.s. kudos from another fan of gensim.
Post reply on HN