Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Search at Slack

slack.engineering

Post reply on HN
Print view View original

11–20 of 94 posts

Re: Search at Slack

#11

Post by g12mcgov »

isabellat wrote: post #3

Hi, I'm one of the authors on the post, happy to answer any questions.

Is there any possibility in the future users will be able to search ALL messages from a channel/private message, ever? It seems like Slack search cuts off after a certain point, and doesn't index into archived messages.

Re: Search at Slack

#12

Post by isabellat »

ankit-singh wrote: post #10

Looks like solr is being used to rank messages using a few features and then re-ranking the top n at application layer using another set of features. This would constrain the search quality as 1) you have no control as to how these two set of features interact and 2) messages ranked low based on first set of features could be highly relevant according to the second set. Another possible approach could have been using…

Great question. We rank in two stages for a number of different reasons. First, it would be too expensive from a performance perspective to rank all of the messages in your corpus. Second, some of the features that we use to rank are much more easily accessed at the application layer. It would require more of an engineering effort to make these signals accessible in SOLR. The first pass which is done in SOLR is a high recall, low precision pass. The second pass through our custom ranker is a high precision pass. It is possible that we would lose some messages that might end up being important in the first pass but it's a tradeoff between performance and accuracy. Hope this helps answer your question.

Re: Search at Slack

#13

Post by isabellat »

g12mcgov wrote: post #11
isabellat wrote: post #3

Hi, I'm one of the authors on the post, happy to answer any questions.

Is there any possibility in the future users will be able to search ALL messages from a channel/private message, ever? It seems like Slack search cuts off after a certain point, and doesn't index into archived messages.

The free version includes search up to 10k of your team's most recent messages. The standard version includes unlimited searchable message archives. https://tinyspeck.slack.com/pricing/slack-for-teams

Re: Search at Slack

#14

Post by Eridrus »

ankit-singh wrote: post #10

Looks like solr is being used to rank messages using a few features and then re-ranking the top n at application layer using another set of features. This would constrain the search quality as 1) you have no control as to how these two set of features interact and 2) messages ranked low based on first set of features could be highly relevant according to the second set. Another possible approach could have been using…

Besides the technical answer already given, this is a pretty standard architecture for search ranking and other problems where your fine grained decisions are too expensive to run on everything.

Re: Search at Slack

#15

Post by isabellat »

iloveluce wrote: post #8
isabellat wrote: post #3

Hi, I'm one of the authors on the post, happy to answer any questions.

Awesome article! The article mentions the signals that the model found were most significant for a message. Curious to know if they're listed by order of significance in the article? And if not was wondering whether one or two signals were predominantly more significant than the rest.

Our top signals are age of the message, whether the message contains a link and whether the message is from the channel that you are currently viewing. We have several different models in production right now but those signals are generally the strongest.

Re: Search at Slack

#16

Post by kusmi »

I've never used Slack because last I remember they don't allow local installation, therefore no access to any documents that were dropped in, is this still the case? I have been using mattermost instead, and wrote a bot which processes all documents uploaded into mattermost, extracts metadata, creates tags and summaries for each, archives the documents in an ECM -categorizing them by their tags, and opens them up for full text solr search. My understanding is that a custom solution like this wouldn't be possible with Slack, or would otherwise require more hacky solutions?

Re: Search at Slack

#18

Post by trafficlight »

leothekim wrote: post #7

This sounds like a step towards making Slack more of a knowledge repository and possibly a wiki replacement. One of my (many) qualms with knowledge repository tools like Google Sites is that searching them is basically useless. Another is that knowledge in these repos becomes stale really quickly. If you can put meaningful information in a Slack post, you can take advantage of the recency-focused nature of chat and s…

I've been searching for a solution to this as well. Our Slack holds an amazing amount of information, but it's really difficult to curate that information efficiently.

It'd be cool to highlight a piece of information and insert it into a wiki-style site.

Re: Search at Slack

#19

Post by amelius »

Open-source desperately needs more search-tool projects.

Lucene/Solr/Elasticsearch are nice, but they need competition, especially outside the Java world.

Post reply on HN