Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A free, privacy preserving, archive of public Discord servers

searchcord.io

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#31

Post by searchcord »

bstsb wrote: post #30

nice project. how are you going to handle the issues involved with breaking Discord's TOS? > "scraping our services without our written consent" additionally, are these pages indexable? i know of other projects (opt-in) that create pages made from user discussion.

> how are you going to handle the issues involved with breaking Discord's TOS?

Not sure. I will solve that problem if and when Discord takes issue with Searchcord.

> additionally, are these pages indexable?

Yes, I would actually like for search engines to index it as their search is much more contextually aware than mine.

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#32

Post by EZ-Cheeze »

I'm in more than a hundred Discord servers. I've been wanting to scrape the members of each of them to discover the people with whom I share the most servers but we're not yet friends. Someone with 10+ would highly likely be a new friend since we'd have a lot of shared niche interests

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#33

Post by searchcord »

EZ-Cheeze wrote: post #32

I'm in more than a hundred Discord servers. I've been wanting to scrape the members of each of them to discover the people with whom I share the most servers but we're not yet friends. Someone with 10+ would highly likely be a new friend since we'd have a lot of shared niche interests

This is something I have been trying to make as a way to learn about graph theory. If I can find a way to make it work efficiently, I will definitely add this.

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#34

Post by 3abiton »

This is an amazing project, I always wonder how much information is lost in those chat apps, not only Discord, but also Telegram. The latter has hude dev community specifically around Android Rom Development, which migrated from forum based XDA to more flexible chat/support platform like Telegram. I wish that also can be searchable without having their client.

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#35

Post by searchcord »

3abiton wrote: post #34

This is an amazing project, I always wonder how much information is lost in those chat apps, not only Discord, but also Telegram. The latter has hude dev community specifically around Android Rom Development, which migrated from forum based XDA to more flexible chat/support platform like Telegram. I wish that also can be searchable without having their client.

Telegram is already heavily monitored and scraped due to the large volume of illegal or extremely controversial activity that happens there. This is something I will look into though, my XDA threads rarely get any replies anymore. Thanks for the suggestion!

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#36

Post by pzmarzly »

nottorp wrote: post #28
searchcord wrote: post #25

Earlier quoted context omitted.

Some very large servers are eligible for what Discord calls "discovery". This makes their data visible without joining the server. You can find a list of those on Discord's site here: https://discord.com/servers

Suggestion: a bot for smaller servers that do want to be archived like a public forum. Their admins could install the bot themselves and perhaps specify what channels they want archived.

Check out Linen https://www.linen.dev/

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#37

Post by jonasdoesthings »

Maybe also exclude messages by bots (e.g. "username has joined the server") from the index to decrease the stalking-potential of your site (99.9% of these bot messages have no informational-value for the index anyways). Currently you can still search for an username and get a subset of servers that the username is in (even if not active) by finding these bot messages.

Re: Show HN: A free, privacy preserving, archive of public Discord servers

#38

Post by searchcord »

jonasdoesthings wrote: post #37

Maybe also exclude messages by bots (e.g. "username has joined the server") from the index to decrease the stalking-potential of your site (99.9% of these bot messages have no informational-value for the index anyways). Currently you can still search for an username and get a subset of servers that the username is in (even if not active) by finding these bot messages.

This is something I have looked into. Unfortunately, every server uses a different format/bot. It might be possible to develop some sort of machine learning classifier to determine if it is a welcome message.
Post reply on HN