Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Linen – Make your Slack community Google-searchable

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Show HN: Linen – Make your Slack community Google-searchable

#11

Post by cheeseblubber »

tomwheeler wrote: post #9

Congratulations on the launch! It looks helpful and easy to use. Two bits of feedback: 1. The threading seems off in places. I looked at the linked osquery site, and there were some top-level items that seemed like replies. Maybe that's just a result of people replying in a channel instead of a thread, though. 2. I see character entities in code sections, instead of their corresponding characters. For example, an XML…

1. Yes it's because someone posted the reply in the channel instead of a thread. I only sync threads at the moment. I have some plans on tweaking the UX to fix that issue and potentially some heuristics to figure out if it is talking about the same subject.

2. Oh good catch. We just rewrote our message rendering so there is some bugs we have to fix.

Re: Show HN: Linen – Make your Slack community Google-searchable

#12

Post by avivo »

cheeseblubber wrote: post #7
forgotmypw17 wrote: post #5

Can you see yourself offering a migration path off of Slack in the future? Your interface is much more accessible not just to Google, but also to this human. I would love to have an option to migrate off Slack with continuity.

Thanks! Not sure of the migration path off of Slack at least in the near term since there is a lot of pull in to Slack since most people in professional setting already is familiar with the UX. I do think there is something there for a layer on top of Slack or Discord where information gets moved. I want to explore making this tool available for internal teams once I have more features around productivity like relate…

I'd be interested in chatting about this at some point. What you are considering re. meaningful aggregation and summarization is connected to my thesis here: https://medium.com/@aviv/when-we-change-the-efficiency-of-kn... (contact info from my profile/website)

Also, this is very useful! Thanks for building it. (Ideally it's bundled with clear info that everything people are writing will be publicly searchable. Perhaps an invisible bot message every month or so.)

Re: Show HN: Linen – Make your Slack community Google-searchable

#14

Post by maxmcd »

cheeseblubber wrote: post #4
yurishimo wrote: post #2

If you can share, how do you retrieve data from the Slack group? Scraping as an invited user? A public slack app? For some communities, that could be a massive amount of webhook traffic! Neat idea :)

Sure! Right now a community admin that adds Linen to your Slack workspace. I have webhooks listening for any message events which gets mirror and saved to the Aurora postgres DB. On install I also have a cronjob that hits the Slack api and pulls the old Slack conversations. For scaling wise the I host the webapp on Vercel where it is backed by lambda functions. I also am using AWS aurora for postgres which in theory…

Since your data is append-only have you considered something like just sticking all of the data in s3?

Might be awkward to deal with things like updating threads, but just editing the s3 file containing the json response for a given thread might work, and then you can skip having a database entirely.

Re: Show HN: Linen – Make your Slack community Google-searchable

#15

Post by treis »

cheeseblubber wrote: post #4
yurishimo wrote: post #2

If you can share, how do you retrieve data from the Slack group? Scraping as an invited user? A public slack app? For some communities, that could be a massive amount of webhook traffic! Neat idea :)

Sure! Right now a community admin that adds Linen to your Slack workspace. I have webhooks listening for any message events which gets mirror and saved to the Aurora postgres DB. On install I also have a cronjob that hits the Slack api and pulls the old Slack conversations. For scaling wise the I host the webapp on Vercel where it is backed by lambda functions. I also am using AWS aurora for postgres which in theory…

Is there a how to or any documentation you followed to do this? I wanted to do something where I consumed all the messages but didn't really see how in my brief documentation reading.

Re: Show HN: Linen – Make your Slack community Google-searchable

#16

Post by hardwaresofton »

blue_cookeh wrote: post #6

I love stuff like this. With so many communities congregating on Discord, Slack etc. we lose a lot of valuable information that would previously have been on something publicly searchable like StackOverflow. A lot of OSS has fallen into this pattern and it can be truly infuriating at times.

It's a bit of a negative comment but I sure wish these communities would rediscover forums. They're not all perfect but I am convinced that Discord and Slack are where valuable information goes to get mixed with gifs and memes then die.

I guess that won't happen until someone starts hosting discourse for free.

Awesome that Linen is trying to build a different future though.

Re: Show HN: Linen – Make your Slack community Google-searchable

#17

Post by cheeseblubber »

maxmcd wrote: post #14
cheeseblubber wrote: post #4

Earlier quoted context omitted.

Sure! Right now a community admin that adds Linen to your Slack workspace. I have webhooks listening for any message events which gets mirror and saved to the Aurora postgres DB. On install I also have a cronjob that hits the Slack api and pulls the old Slack conversations. For scaling wise the I host the webapp on Vercel where it is backed by lambda functions. I also am using AWS aurora for postgres which in theory…

Since your data is append-only have you considered something like just sticking all of the data in s3? Might be awkward to deal with things like updating threads, but just editing the s3 file containing the json response for a given thread might work, and then you can skip having a database entirely.

Oh interesting. I think like you said updates might be a little bit tricky as well as search. I'm using full text search from Postgres so that would be tough on S3.

I'm doing some aggressive caching with Vercel and Nextjs which essentially turns in to putting the data in S3.

Re: Show HN: Linen – Make your Slack community Google-searchable

#18

Post by charcircuit »

treis wrote: post #15
cheeseblubber wrote: post #4

Earlier quoted context omitted.

Sure! Right now a community admin that adds Linen to your Slack workspace. I have webhooks listening for any message events which gets mirror and saved to the Aurora postgres DB. On install I also have a cronjob that hits the Slack api and pulls the old Slack conversations. For scaling wise the I host the webapp on Vercel where it is backed by lambda functions. I also am using AWS aurora for postgres which in theory…

Is there a how to or any documentation you followed to do this? I wanted to do something where I consumed all the messages but didn't really see how in my brief documentation reading.

The meat of it would presumably use:

https://api.slack.com/methods/conversations.list

https://api.slack.com/methods/conversations.history

https://api.slack.com/events/message

Re: Show HN: Linen – Make your Slack community Google-searchable

#19

Post by maxmcd »

cheeseblubber wrote: post #17
maxmcd wrote: post #14

Earlier quoted context omitted.

Since your data is append-only have you considered something like just sticking all of the data in s3? Might be awkward to deal with things like updating threads, but just editing the s3 file containing the json response for a given thread might work, and then you can skip having a database entirely.

Oh interesting. I think like you said updates might be a little bit tricky as well as search. I'm using full text search from Postgres so that would be tough on S3. I'm doing some aggressive caching with Vercel and Nextjs which essentially turns in to putting the data in S3.

Ah didn't think about search, makes sense. Yeah agree the benefits caching are roughly equivalent to storing in s3.

Re: Show HN: Linen – Make your Slack community Google-searchable

#20

Post by cheeseblubber »

treis wrote: post #15
cheeseblubber wrote: post #4

Earlier quoted context omitted.

Sure! Right now a community admin that adds Linen to your Slack workspace. I have webhooks listening for any message events which gets mirror and saved to the Aurora postgres DB. On install I also have a cronjob that hits the Slack api and pulls the old Slack conversations. For scaling wise the I host the webapp on Vercel where it is backed by lambda functions. I also am using AWS aurora for postgres which in theory…

Is there a how to or any documentation you followed to do this? I wanted to do something where I consumed all the messages but didn't really see how in my brief documentation reading.

Like @charcircuit said those were the meat of it. Main part is you have to iterate through conversation.history api. The syncing portion definitely is the most complicated part of the code base.
Post reply on HN