tldr (4,473 words); Search is a classic NLP problem replete with all of natural languages'
wide-ranging input and evaluation vagaries. I admire Russian get-to-the-point
bluntness in most contexts, but for whatever cultural reason, their writing is
always Tolstoyesque.
What software engineers should know about search (2017)
81–90 of 132 posts
Re: What software engineers should know about search (2017)
#82Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…
I will very quickly stop using a given website if it doesn't provide a good, on-site search. If I'm looking for information that I know exists on a particular website, I see no reason to involve a third-party search engine in the equation at all. Granted, a lot of sites out there that provide a search function do not necessarily need one, and would instead benefit from an organized, hierarchial index. With these type…
Given the state of search functionality built into websites I use often I can only imagine you have a very heavily curated list.
In my experience so many sites have searches that return bad results, restrict your ability to see a list of results due to some poorly thought through typeahead functionality, or simply do not work without third party scripts and cookies that I often resort to "site:" in ddg or worse, google.
I find hierarchical indexes only of marginal use when looking for information. I can only know where in the hierarchy something is if I'm very familiar with the website - because many things could logically be in more than one place.
Re: What software engineers should know about search (2017)
#83Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…
I will very quickly stop using a given website if it doesn't provide a good, on-site search. If I'm looking for information that I know exists on a particular website, I see no reason to involve a third-party search engine in the equation at all. Granted, a lot of sites out there that provide a search function do not necessarily need one, and would instead benefit from an organized, hierarchial index. With these type…
> organized, hierarchial index
Yes!!
So many blogs without a list of posts in chronological order. So many sites without a list of pages. So annoying.
Re: What software engineers should know about search (2017)
#84Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…
This is true in principle, but in practice, the use of 3rd party search has died down over the last decade. This is not the phenomenon of a superior way winning out. Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search base…
> Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search based UI, then you'll probably roll your own.
Most stores have garbage search and even worse filtering. >98 % of stores have only categories for filtering and only trash like "price" for sorting. In the 2 % of stores that can actually do faceted search the number of facets is too small (e.g. shoe stores which only have a "size" facet plus categories) or the data in the facets is garbage. This is likely a big contributor to sites like Geizhals existing, whose sole purpose is to offer decent search and filtering.
Re: What software engineers should know about search (2017)
#85I often come back to the Relevant Search book, by Doug Turnbull and John Berryman. I'm sure some of the examples are a little dated now, but most of the advice and approach is still sound and it's a great tour through all the things you need to consider to build a great search experience. https://www.manning.com/books/relevant-search
Doug Turnbull here - Thank you! :)
Re: What software engineers should know about search (2017)
#86Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…
I use DuckDuckGo's bangs[0] to search, say Wikipedia (!w, !wf), Word Reference (!wref, !wrfe), Hacker News (!hn). They are incredibly useful and using Google would be a waste of time for such queries. I'm glad these sites implemented search.
Re: What software engineers should know about search (2017)
#87Earlier quoted context omitted.
This is true in principle, but in practice, the use of 3rd party search has died down over the last decade. This is not the phenomenon of a superior way winning out. Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search base…
> Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search based UI, then you'll probably roll your own. Most stores have garbage search and even worse filtering. >98 % of stores have only categories for filtering and only tras…
I don't disagree.
If you have bad search, and search is a primary feature, then your ux sucks. I just don't think "just use google" is a viable alternative to making search good.
If you're making an online store, search is probably important, and a major determinant of how well the software works. The best way to handle that is with search that doesn't suck. It may be hard, but that's the job. Fair point that search is not trivial to do well, but that doesn't mean it isn't the job.
Re: What software engineers should know about search (2017)
#88Not sure who needs to know this but scribe.rip is a relatively new alternative reader for Medium. The original article is at: https://medium.com/startup-grind/what-every-software-enginee... HN's special treatment of medium.com links doesn't apply to scribe ones.
That explains why all the link texts are weirdly misaligned. Do the authors have any input or does it just scrape Medium for content?
What would they have input about?
Re: What software engineers should know about search (2017)
#89Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…
If this is your situation there's a nice site search builder for DDG here: https://ddg.patdryburgh.com/
I use that for my static site as it means I don't need to introduce any moving parts. Probably not the best search experience but it gets the job done
Re: What software engineers should know about search (2017)
#90I won't repeat what others have said about advances in natural language processing since 2017, but it's true that it's a solved problem if your problem isn't "perfect search" but a more realistic "excellent search".
> Use existing technologies first: As in most engineering problems, don’t reinvent the wheel yourself. When possible, use existing services or open source tools. If an existing SaaS (such as Algolia or managed Elasticsearch) fits your constraints and you can afford to pay for it, use it.
I work at an AI search company (Relevance AI) and even we see that InstantSearch.js is all some people need in terms of UI. We created a version of it that uses our NLP-backend but is still the same Algolia components on the frontend: https://www.npmjs.com/package/@relevanceai/instant-relevance
The reason was because those components work. Think these days you'd need to ask a lot of questions before completely rolling your own UI or NLP handling.