Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What software engineers should know about search (2017)

scribe.rip

Post reply on HN
Print view View original

91–100 of 132 posts

Re: What software engineers should know about search (2017)

#91

Post by kristopolous »

woolion wrote: post #51
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

Is there any actual evidence that might be true? I encountered this argument many times, but only from programmers (who thought it was to difficult because of their tech stack), never from users. As a user, there is no site that does not have search, no matter how good it is, that does not feel quite horrible because of that. Even the search by Google feel horrible -- romhacking comes to mind. It is also frustrating…

The statement is intentionally hedging and qualitative. Some people do the usage pattern, some don't.

The advocacy is to push back on search as a difficult and often unnecessary problem to solve.

The caveat is my increasingly toxic pattern to push back on almost everything as unnecessary. It's probably overly antagonistic.

I'm a techno pessimist programmer. I didn't understand this attitude when I was younger but then again people often become that which they fail most to understand

Re: What software engineers should know about search (2017)

#92

Post by wongarsu »

formerly_proven wrote: post #84
netcan wrote: post #46

Earlier quoted context omitted.

This is true in principle, but in practice, the use of 3rd party search has died down over the last decade. This is not the phenomenon of a superior way winning out. Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search base…

> Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search based UI, then you'll probably roll your own. Most stores have garbage search and even worse filtering. >98 % of stores have only categories for filtering and only tras…

Ninety percent of ecommerce sites have terrible search because ninety percent of everything is crap [1].

That doesn't mean that having a great search can't help your shop get repeat customers, just like good service, fast shipping or a good checkout flow can.

1: https://en.wikipedia.org/wiki/Sturgeon%27s_law

Re: What software engineers should know about search (2017)

#93

Post by pc86 »

BatteryMountain wrote: post #47
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

This only counts for public sites with semi-static information.. I've been on plenty of internal/LOB projects where the filtering/searching/slicing of data and exporting to various formats are one of the core features of the application, where the data will also be different as time marches forward (non-static sites). Thousands of non-tech people rely on those kinds of features to do their job. Oh, and some of them a…

Well they said in the very first sentence it only applies if you're indexed by search bots. Internal apps (typically) aren't accessible at all, and quickly-changing data isn't indexed in time to be relevant.

Re: What software engineers should know about search (2017)

#94

Post by kristopolous »

chmod775 wrote: post #45
tda wrote: post #41

Earlier quoted context omitted.

I have never ever been ale to find anything on Wikipedia with search, except if I know the title of the article I'm looking for. But as I often do want the information from Wikipedia, I just search , and it takes me straight to where I want to be, whether I'm using DDG or Google. Works so much better than !w

Lately I had to scroll down on Google search results a lot to find the relevant Wikipedia article, it often being somewhere below some irrelevant images, followed by a completely unasked for and irrelevant map (why in gods name would I care for where the nearest factory for a product is?), some random blogspam, and ads. It used to be that you reliably had the Wikipedia article at the top of your results to provide co…

I've lost faith in the ability of the Google approach.

Their results seem to have been turning to trash but then again, so has everyone else's.

There's a few explanations. Easiest one is me, I'm getting dumber with age or have changed my standards. Second one is they are all using similar approaches and SEO'ing has ruined search. Third is Google sets "the standard" and the other engines tweak themselves to follow the goog, regardless of results.

Reality is it's probably all 3 and a few more that I haven't thought of

Re: What software engineers should know about search (2017)

#95

Post by mistrial9 »

marginalia_nu wrote: post #53
kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

Wikipedia actually doesn't want to be indexed by third party search engines. Mediawiki is a heap of underoptimized PHP garbage so pages are (expensively) rerendered every time you visit them.

> underoptimized PHP garbage

rather uncharitable description - that PHP implements dozens of interesting features and recall. It is true that it is a mess, however.

Re: What software engineers should know about search (2017)

#96

Post by danpalmer »

dotancohen wrote: post #76
danpalmer wrote: post #74

The biggest thing I've learned about search is to be very precise about WHAT is being searched. In my experience most people are very loose with this, but by being really precise you can skip past a lot of bad search experiences. Take clothing search, "blue t-shirt", what is the search space? Most engineers would naively shove the name/description into Elastic Search and then be surprised when it doesn't work. Why? B…

One strategy for this is to parse the search string for facets. Presumably there would be a facet for "t-shirt" and another facet for "blue", modulo synonyms and typos. Selecting those facets should give very good results, even without hitting the full-text search.

Yes that's a good option, but it still requires that you've got the facets in the index. I'm surprised at how many people say "just use postgress full-text search" without thinking through what information will actually be made available to search, for example.

Re: What software engineers should know about search (2017)

#97

Post by bjoernbu »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

If I think of myself as a user, that is spot on for information/entertainment, but far from the truth for products. The most extreme examples being everyone using "$THING Wikipedia" but nobody I know using "$THING Amazon" in Google search

Re: What software engineers should know about search (2017)

#98

Post by whakim »

I want to plug Algolia a little bit. For small teams, the results are truly incredible, and it provides a ton of features out-of-the-box without a lot of development time. I've seen small teams struggle mightily with both ElasticSearch and Solr; things usually start off OK, but results go downhill as the search needs get more complicated (indexing multiple kinds of documents; adding more and more different kinds of data to the index and trying to weight things properly; dealing with tradeoffs such as relevancy vs recency; etc.). In the future I would shy away from such powerful tools unless I knew I had sufficient engineering resources to dedicate to search and my problem was complex enough to merit it.

Re: What software engineers should know about search (2017)

#99

Post by forwidur »

Author of the article here: thanks for re-posting this! The article seems to be still relevant to many even 4 years later, which surprises me, given how quickly everything is changing in the field.

However, I personally am now focused (and bullish) on DNN-based semantic search. Having built several search experiences based on it I'm convinced it is the future.

Re: What software engineers should know about search (2017)

#100

Post by fatnoah »

kristopolous wrote: post #2

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

>They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well.

I'm someone who does this, and I do it because I've found that it either works better than a site's specific search and/or it's so much faster to go to Google and search than to try to find the search functionality of the specific site.

Post reply on HN