Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What software engineers should know about search (2017)

scribe.rip

Post reply on HN
Print view View original

51–60 of 132 posts

Re: What software engineers should know about search (2017)

#51

Post by woolion »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

Is there any actual evidence that might be true? I encountered this argument many times, but only from programmers (who thought it was to difficult because of their tech stack), never from users.

As a user, there is no site that does not have search, no matter how good it is, that does not feel quite horrible because of that. Even the search by Google feel horrible -- romhacking comes to mind. It is also frustrating because some instances of title searches are notoriously unindexable, while a nice site search lets you narrow the context through tags or model specific attributes (system, genre, ...).

Re: What software engineers should know about search (2017)

#52

Post by marginalia_nu »

I think more software engineers should do themselves the favor of dabbling in search engine design.

It really is the gift that keeps on giving in terms of interesting problems, you get everything from large scale graph processing, esoteric data structures, down to bit twiddling optimization, language processing, dealing with seriously messy and sometimes adversarial real world data. Everything is challenging, but none of it is impossible, even as a solo project.

Re: What software engineers should know about search (2017)

#53

Post by marginalia_nu »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

Wikipedia actually doesn't want to be indexed by third party search engines. Mediawiki is a heap of underoptimized PHP garbage so pages are (expensively) rerendered every time you visit them.

Re: What software engineers should know about search (2017)

#54

Post by EE84M3i »

sydthrowaway wrote: post #43
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

An interesting example is the search query “ reddit”. Without the ‘reddit’ qualifier, the results are nearly always spammy and useless (as much as modern Reddit tries to compete here notwithstanding)

Its pretty often that I do a search, get bad results, then add reddit and get good results, so I hope some data analyst in Google sees those as opportunities to improve.

Re: What software engineers should know about search (2017)

#55

Post by ChrisMarshallNY »

That’s a useful and well-written article. It was written 4 years ago, though, and NLP has improved significantly in that time (in English).

My experience using natural-language search queries has given me a set of expectations, as a user, that set a high bar.

I’ve written a fairly substantial backend for a search, and most of my original frontend code has been ripped out and replaced with new frontends, by now. I feel that the new frontends are an improvement, but we still have a fairly strictly-guided “guardrail” search.

Re: What software engineers should know about search (2017)

#56

Post by karterk »

carlreid wrote: post #49
karterk wrote: post #12

A problem close to my heart. Good search is certainly still too difficult to pull off for small teams, and this was one of my motivations for building and open sourcing Typesense[1]. Most people think of search and immediately think of large data sets, but the problems that plague smaller datasets are equally interesting. It's less about performance and more about relevance. For e.g. searching across multiple fields…

Typesense has been great for us so far. Easy to set up, works great with the simple queries that we need. Lots of great additional functionality on top of search, for example security, with scoped API keys and the likes that we're looking forward to making use of.

Thank you for your kind words. We are just getting started :)

Re: What software engineers should know about search (2017)

#57

Post by sriram_malhar »

kiryin wrote: post #36
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

I will very quickly stop using a given website if it doesn't provide a good, on-site search. If I'm looking for information that I know exists on a particular website, I see no reason to involve a third-party search engine in the equation at all. Granted, a lot of sites out there that provide a search function do not necessarily need one, and would instead benefit from an organized, hierarchial index. With these type…

Would it bother you if foo.com took your query and send "query site:foo.com" to google? The results would be foo.com specific.

Re: What software engineers should know about search (2017)

#58

Post by RicoElectrico »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

You're totally right. Especially forum search engines are terrible. This is the best bang-for-buck solution.

If someone has any idea why forum search experience is so bad and know how to improve it, please chime in. I have my hunches, but let's not get ahead of myself.

Re: What software engineers should know about search (2017)

#59

Post by lobocinza »

kiryin wrote: post #36
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

I will very quickly stop using a given website if it doesn't provide a good, on-site search. If I'm looking for information that I know exists on a particular website, I see no reason to involve a third-party search engine in the equation at all. Granted, a lot of sites out there that provide a search function do not necessarily need one, and would instead benefit from an organized, hierarchial index. With these type…

I agree with you but many sites use a third-party search engine to provide on-site search functionality. e.g. Algolia's DocSearch

Re: What software engineers should know about search (2017)

#60

Post by mro_name »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

or do it like the w3c - make a search form with action https://www.w3.org/Help/search?q=xslt.

Privacy may be a thing, however.

Post reply on HN