31–40 of 57 posts
I think search problems are a ton of fun, because there are a lot of opportunities for creative solutions to problems. One thing I'd add is a book called "Relevant Search" - I have a hobby project to search talks, and the book helped me out a ton ( https://www.findlectures.com ).
I think somewhere along the way work on search just devolved into handling millions of queries at a time. It could have been so much better. If you ask a "search expert" today what's he is trying to fix, he will say something related to scaling. If you asked an expert in the 80's or 90's they would talk about query complexity and NLP i.e. Who were the four semi-finalists of last years Wimbledon? And you would get bac…
Ex-Google search engineer here, now using hosted ElasticSearch extensively in my startup. This is a really good overview. If there's one part I want to highlight, it's that you should expect to spend a lot of time fine-tuning your ranking function for your particular product & corpus . The default ElasticSearch ranking function kinda sucks. It was changed in ES 5.0 to Okapi BM25, which is the current academic state-o…
what does your startup do?
Still under development, but drop me an e-mail if you (or anyone else reading this) is interested in beta-testing. I'm starting out e-mail first, so the initial UI is just that you get a daily digest of links & snippets to threads related to your interests.
I think somewhere along the way work on search just devolved into handling millions of queries at a time. It could have been so much better. If you ask a "search expert" today what's he is trying to fix, he will say something related to scaling. If you asked an expert in the 80's or 90's they would talk about query complexity and NLP i.e. Who were the four semi-finalists of last years Wimbledon? And you would get bac…
Well, I hear a lot of people complaining that the results on DuckDuckGo are still worse than on Google, even though both search-engines produce results within a second. And these are people that really want to quit using Google for privacy reasons. I never hear people complaining that a search is slow. So I do think that search-quality is where the competition is happening. Edit: But, I agree, we don't often see any…
Earlier quoted context omitted.
what does your startup do?
Cross between an RSS-reader and a search engine - it lets you subscribe to a topic instead of a site, so that you don't have to manage hundreds of subscriptions, and if a new discussion pops up somewhere on the web that's relevant to your interests, it'll find it and let you know. As forums get boring or off-topic and new ones spring up, it adjusts automatically, so you don't need to do the "Does anyone know of other…
Another recommended book that was not mentioned in the post: Search Engines: Information Retrieval in Practice https://www.amazon.com/Search-Engines-Information-Retrieval-...
Just. Great.
I used to work on textual Information Retrieval (which is quite related to search). One of the key contributions of my PhD thesis was a statistical method to extract concepts from texts [0]. Surprisingly enough, with those concepts, Tf-Idf [1] was quite good to extract keywords from documents which allowed us to build document descriptor tables which can eventually be used for document search [2]. We also built a sma…