Live data from Hacker News

Ask HN: Is 'search' a solved problem?

news.ycombinator.com

1–10 of 125 posts

Ask HN: Is 'search' a solved problem?

#1
I remember the time (not so long ago) when 'search' seemed to be the hottest topic in the industry. We had the rise of Google and competitors. There were search startups. Open source projects like Lucene and Solr were in the news. There were books published, blogs, conferences..

And now it seems that the industry have moved on. There is a million papers/books/blogs/online courses/video lectures/meetups about ML/AI, but I can't seem to find anything current on search.

What are the good resources to learn the fundamentals of search and keep up with the current happenings in that space? Not SEO, but more from the computer science/engineering point of view?

Re: Ask HN: Is 'search' a solved problem?

#2
It depends what you are looking for. I have the hardest time searching for laptops that meet all of the specs I want, for instance.

Takes days.

Minimum screen brightness. 1440p touch. Needs USB-A. 8th gen quad core. 20+ watt TDP. etc. Not too thin, not too thick. decent graphics.

Re: Ask HN: Is 'search' a solved problem?

#5
Lucene/Solr, Elastic Search and Algolia did a great job creating search tools and services and this extinguished the thirst of the masses. I don't think it's a solved problem, it's just a problem that has commercially viable solutions. When it comes to resources, I've found valuable knowledge in Lucene/Solr forums and mailing lists, back in the day. It's worth a read.

Re: Ask HN: Is 'search' a solved problem?

#6

It depends what you are looking for. I have the hardest time searching for laptops that meet all of the specs I want, for instance. Takes days. Minimum screen brightness. 1440p touch. Needs USB-A. 8th gen quad core. 20+ watt TDP. etc. Not too thin, not too thick. decent graphics.

I think this issue resides with the OEM's not providing information. Search or indexing cannot solve this problem. I think you'll have to watch plenty of YouTube influencers in laptop domain to find what you're looking for.

Re: Ask HN: Is 'search' a solved problem?

#7
I don't see 'search,' aka information retrieval (IR), as a solved problem. I went to the last SIG IR conference in Tokyo, and yes, the heavy hitters in the field were there to promote and present their latest research. It is no doubt a very active research field using machine learning techniques. Reading the papers published could give a view of the (academic) state of the art.

Whether it is a good business strategy to challenge Google head-on is another question. Whether there is a sure way to learn and engineer said systems on extremely large scales is another question.

Re: Ask HN: Is 'search' a solved problem?

#8

It depends what you are looking for. I have the hardest time searching for laptops that meet all of the specs I want, for instance. Takes days. Minimum screen brightness. 1440p touch. Needs USB-A. 8th gen quad core. 20+ watt TDP. etc. Not too thin, not too thick. decent graphics.

In the EU you can use https://geizhals.eu/?cat=nb which is pretty decent. I'm using that regularly to check options.

Is something like this not available in the US/other territories? If not: here is your opportunity :-D

Re: Ask HN: Is 'search' a solved problem?

#9
To break into this space, I would recommend starting with a subset that people want to search. Like the facebook model of only being available for some colleges.

Some ideas:

- Only academic papers

- Only news sources

- Only hacker topics

- Only financial topics

- Only small bloggers

- Only literal keyword search which Google discontinued

Get traction in that domain, then build out from there.

Re: Ask HN: Is 'search' a solved problem?

#10
Not really, no. Some parts of search (Internet) have been cornered/solved by big players (e.g. Google), but many other parts (e.g. search in Intranets) are still an open problem. No one has found a solution that's as simple as PageRank for Intranets. No one has found a solution for the "the author of half of these documents is the intern who wrote the template"-problem and many other things. There are good products out there, but a "Google for Non-Internet" is still far away.

p.s.: All the bad searches on the various products I use or the websites of some companies make me almost want to get back into search. Because even for things that ARE solved in the technical sense there seems to be no "out of the box" solution or people would use it.

edit: To expand a bit - all the search solutions I've seen which weren't for Internet search were more or less bespoke, so you needed a project to get something decent. Sure, you can install a plain Lucene/Solr, but Lucene/Solr cannot understand the way your data works, which parts are important or if you want to show results which are older further down (or not!). They have decent defaults for the "common case", but you have to tune them for every customer/installation for good results and that makes it non-scalable. And being scalable without effort is usually one of the requirements people have for something to be "solved".

Post reply on HN