Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Answer Overflow – Indexing Discord content into the web

answeroverflow.com

Post reply on HN
Print view View original

21–30 of 99 posts

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#21

Post by thrashh »

apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

Discord is a chatroom first. What non-enterprise chat comes with archives?

A forum is totally different.

And even then, forums weren’t designed to be archived from the start. People just wrote web crawlers and search engines.

(I know Discord has some forum-like functionality now but the point stands.)

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#22

Post by madeofpalk »

apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

Discord has 'indexing' and search, just like how Slack does. It's just not on the public & open web - only searchable inside of Discord.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#23

Post by thunky »

apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

What I wonder is why would anyone that cares about archiving/search would choose to use Discord?

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#24

Post by michaelmior »

Kiro wrote: post #13
apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

It makes no sense to index the vast majority of content. You would need to cherry pick really hard among all the noise to find the stuff worth putting online.

I would argue it makes no sense to index the vast majority of content without good search. If your search is good enough, you can index everything and then surface only the good stuff at query time.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#25

Post by hutzlibu »

thunky wrote: post #23
apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

What I wonder is why would anyone that cares about archiving/search would choose to use Discord?

Apparently because it is very easy to setup and offer a place where people can join.

More and more open source projects are using it and I don't really like it, but what easy alternatives can you recommend to them?

Genuine question, as it is an open issue for me. I want to focus on my project, not setting up and maintain a forum, mailing lists, etc. on top of that.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#26

Post by easygenes »

TeMPOraL wrote: post #15
easygenes wrote: post #6

While I see the value here, I don't really think most Discord communities are appropriate to be indexed. It breaks the whole cozy web aspect of it. [1] [1] https://maggieappleton.com/cozy-web

The "cozy web" is out of control these days. A lot of social utility is lost by default because everyone uses Whatsapp and Discord and other such information black holes, places where knowledge goes to die. It's OK if you're using these to chat with your family or friends, but it's kind of... less OK, when every open source project these days, including major programming languages, tells you to join their Slack or Di…

I was an active participant of the 90s web, and in fact a lead editor and forum moderator for a popular turn of the millennium news site, so I understand the frustration you're sharing.

That said, I'd argue it's not the "cozy web" that's out of control, but instead the "dark forest" that has forced the creation of the cozy web. The cozy web is the only bastion of the internet left where there's still some semblance of the pay it forward community aspect of the early web.

Yes, it is at the cost of not being indexed, but it's the only way of having the genuine sorts of conversations and creation with people of shared interests that typified the early web now.

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#28

Post by jacooper »

hutzlibu wrote: post #25
thunky wrote: post #23

Earlier quoted context omitted.

What I wonder is why would anyone that cares about archiving/search would choose to use Discord?

Apparently because it is very easy to setup and offer a place where people can join. More and more open source projects are using it and I don't really like it, but what easy alternatives can you recommend to them? Genuine question, as it is an open issue for me. I want to focus on my project, not setting up and maintain a forum, mailing lists, etc. on top of that.

Matrix, but that doesnt really fix the searchability problem

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#29

Post by easygenes »

rhyssullivan1 wrote: post #9
easygenes wrote: post #6

While I see the value here, I don't really think most Discord communities are appropriate to be indexed. It breaks the whole cozy web aspect of it. [1] [1] https://maggieappleton.com/cozy-web

Most Discord communities aren't meant to be indexed I agree! Thanks for linking that article it was interesting to read There's lots that have support channels though for programming libraries, for games, etc and having all of that content locked away can be really damaging. One of the interesting things I've noticed is when a community for a more niche game / programming library joins Answer Overflow, they often sho…

Thanks for the thoughtful response. Glad to see this is something you care about preserving.

Good to see you're careful to only share particular channels.

I have more thoughts on marketing this and also on guidelines for server administrators implementing search indexing. For marketing, most importantly, it could be good to make it clear you're focused on selective sharing only of channels which it would be a public good to make indexable. For administrator guidelines, most importantly, I think there should be several measures to ensure that users are aware of and agree to having their communications in particular channels publicly indexed.

I ran this by GPT-4 for some more context and detail. [1]

I think with measures like this we may be able to realize the good of indexing without going too far to driving away the safety of the walled garden aspect of Discord.

As an aside, for users of existing Discords, I encourage you to learn to use the search features built into Discord. Discord itself indexes servers and the search has good filtering functionality. I suspect if you already know which Discord server has the information you're looking for, you'll have a better experience with the internal search than trying to lean on Google.

If you want to do better than the internal search, perhaps creating a vector store of the channel and setting up an AI chat application in front of it would be a solution.

[1] https://chat.openai.com/share/254632c2-c25b-4299-88c9-2ce49e...

Re: Show HN: Answer Overflow – Indexing Discord content into the web

#30

Post by dancemethis »

rhyssullivan1 wrote: post #7
apignotti wrote: post #5

Genuine question: I love Discord, but how on earth is it possible that such functionality was not built-in to begin with? I really don't understand how the need for indexing and search was overlooked.

I think it's due to how Discord evolved as a platform Discord start as "your private place for your friends to talk" during a time where there were a lot of privacy issues with other communication methods. Then as it grew beyond this scope of being a private place for friends, it would have been good for indexing to be added but indexing a normal text channel is really hard since you don't know where the conversation…

It's beyond ironic that Discord called itself that, being the successor to OpenFeint and its privacy lawsuit scandal, and being proprietary.

Now it is one of the most privacy-hostile AND preservation-hostile platforms around.

Post reply on HN