Live data from Hacker News

What software engineers should know about search (2017)

scribe.rip

41–50 of 132 posts

Re: What software engineers should know about search (2017)

#41

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

I have never ever been ale to find anything on Wikipedia with search, except if I know the title of the article I'm looking for. But as I often do want the information from Wikipedia, I just search , and it takes me straight to where I want to be, whether I'm using DDG or Google. Works so much better than !w

Re: What software engineers should know about search (2017)

#42
post #21
post #18

Earlier quoted context omitted.

As a long time lurker of your work, and someone who works on a fair number of search engines myself - I'm curious, how well does TypeSense handle code search (punctuation, etc.)?

Finally added support for indexing and/or separating on specific symbols in the ongoing pre-release builds. With that, I think Typesense should be able to handle code search, but I have never tried to index and search on code myself :)

Cool! I’d be keen to try it out sometime if you have docs on how I’d get it to index specific symbols

Re: What software engineers should know about search (2017)

#43

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

An interesting example is the search query “ reddit”.

Without the ‘reddit’ qualifier, the results are nearly always spammy and useless (as much as modern Reddit tries to compete here notwithstanding)

Re: What software engineers should know about search (2017)

#44

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

That does make you dependent on Google not starting to suck. For the first time last week I had the experience of Googling a programming question and realizing the results were so bad I should probably try searching StackOverflow directly. I did and got twice as many hits.

Re: What software engineers should know about search (2017)

#45
post #41

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

I have never ever been ale to find anything on Wikipedia with search, except if I know the title of the article I'm looking for. But as I often do want the information from Wikipedia, I just search , and it takes me straight to where I want to be, whether I'm using DDG or Google. Works so much better than !w

Lately I had to scroll down on Google search results a lot to find the relevant Wikipedia article, it often being somewhere below some irrelevant images, followed by a completely unasked for and irrelevant map (why in gods name would I care for where the nearest factory for a product is?), some random blogspam, and ads.

It used to be that you reliably had the Wikipedia article at the top of your results to provide context and basic information in case you didn't know what your search term means, you could expect it to be there if it exists. Now you have to waste mental energy hunting it down, which is a waste if there's no article at all.

Re: What software engineers should know about search (2017)

#46

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

This is true in principle, but in practice, the use of 3rd party search has died down over the last decade. This is not the phenomenon of a superior way winning out.

Either search is not an important feature, and a suboptimal, DIY implementation that looks OK is good enough. Or, search is a primary feature and then you need control over it. IE, if you have an online store, travel site or dating app with a search based UI, then you'll probably roll your own.

There are cases where google really is the best way. As you say, stackoverflow, wikipedia & such. Even so, you'll eventually roll your own. Spolsky original UI concept totally leaned on Google for search, but SO still has its own.

At some point, you'll need results to take inventory into account or make autocomplete smarter about tags... and now the headache is yours anyway.

It would have been cool if the web had really developed into the hopeful, "semantic web era" where this kind of approach works. That didn't happen. Half the game is over control, and controlling UIs matters the most.

Also, the power of pagerank has dwindled as the web itself changed. Links aren't what they used to be. That makes Google relatively worse at what it was once best at. Google search as a whole is much, much richer but most of what makes Google good today has less to do with your use case anymore.

TLDR, Maybe google search works for searching wikipedia, which is perfectly loyal to the original WWW concept. Even they have a DIY search. If you work at Reddit though, and your search sucks, google will not fix this. Your search will just suck, and your app will be less usable.

Re: What software engineers should know about search (2017)

#47

Biggest advice I can give is you probably don't need search if you're indexable by search bots. No really. Look over people's shoulders sometime. They'll just go to Google and type in their search followed by terms such as Wikipedia, imdb, Stackoverflow, YouTube, Bandcamp, Amazon, eBay, Yelp... all sites that spent a lot of time on their search and have done quite a decent job. Oh well. So unless you really need it f…

This only counts for public sites with semi-static information.. I've been on plenty of internal/LOB projects where the filtering/searching/slicing of data and exporting to various formats are one of the core features of the application, where the data will also be different as time marches forward (non-static sites). Thousands of non-tech people rely on those kinds of features to do their job. Oh, and some of them are behind firewalls/proxies/vpn and never see the light of day on the public internet, so no search crawlers can see them - and even if they could, they are not optimized for crawlers and/or too dynamic.

Re: What software engineers should know about search (2017)

#48
post #42
post #21

Earlier quoted context omitted.

Finally added support for indexing and/or separating on specific symbols in the ongoing pre-release builds. With that, I think Typesense should be able to handle code search, but I have never tried to index and search on code myself :)

Cool! I’d be keen to try it out sometime if you have docs on how I’d get it to index specific symbols

Yes, please see here: https://github.com/typesense/typesense/issues/122#issuecomme...

The latest RC build is `0.22.0.rcs18`.

Re: What software engineers should know about search (2017)

#49
post #12

A problem close to my heart. Good search is certainly still too difficult to pull off for small teams, and this was one of my motivations for building and open sourcing Typesense[1]. Most people think of search and immediately think of large data sets, but the problems that plague smaller datasets are equally interesting. It's less about performance and more about relevance. For e.g. searching across multiple fields…

Typesense has been great for us so far. Easy to set up, works great with the simple queries that we need.

Lots of great additional functionality on top of search, for example security, with scoped API keys and the likes that we're looking forward to making use of.

Post reply on HN