Viewing profile — searchcord

This all depends on when the message is scraped. The scraper does not go back and check for edits/deletions. This is something that while could be added, would be incredibly ineffi…

> The sheer audacity here is quite something. You're stating people can't use your scraped data for commercial purposes "without permission," while your entire project is built on …

lol. It's only on the homepage, not on any content pages.

This is something I have looked into. Unfortunately, every server uses a different format/bot. It might be possible to develop some sort of machine learning classifier to determine…

Telegram is already heavily monitored and scraped due to the large volume of illegal or extremely controversial activity that happens there. This is something I will look into thou…

This is something I have been trying to make as a way to learn about graph theory. If I can find a way to make it work efficiently, I will definitely add this.

> how are you going to handle the issues involved with breaking Discord's TOS? Not sure. I will solve that problem if and when Discord takes issue with Searchcord. > additionally, …

This is something I have already completed but have not finished bug testing. The bot also includes functionality to recover any server in case it was nuked/wiped and Searchcord ha…

Probably not. Discord will aggressively captcha you and every server has a different implementation of verification. It might be possible with a captcha solver and then some LLM to…

Hey, Thanks for your suggestions. > 1) I'd suggest anonymizing the usernames / author ids to something more privacy friendly such as how some image sites were generating 3-4 random…

Some very large servers are eligible for what Discord calls "discovery". This makes their data visible without joining the server. You can find a list of those on Discord's site he…

Hey, This is interesting, I somehow missed this. Unfortunately, those are not full text searchable. Maybe I will download them and import them into Searchcord, with proper credit o…

I hope Searchcord helps you! <3

Hey, This is absolutely something I want to do, but at the guild level. The database itself is over 13TB which is much to large to create regular exports of. I will probably provid…

Hey, Not exactly. Attachments are only fetched from Discord as the user requests them. This means that the vast majority of attachments are never stored on my server. Right now, I …

Hey, Thanks for your suggestions. However, this does not work for a few reasons: 1. Joining servers is protected by increasingly difficult to solve captchas that have no commercial…

Sure, but there's a few petabytes of attachments and over 63 billion messages. Feel free to use the API.

Thanks for your feedback! <3

I already own the hardware, so I only pay for colocation and transit. It's probably a lot less than you think. I hope to find some way to monetize it, but it is cheap enough that I…

Hey, Thanks for your feedback. For software, I use ScyllaDB and Elasticsearch. It's split across 6 physical nodes (8 including the CDN). Data collection is handled using standard u…

Hey HN! I have been working on this project for a while, and I think this solves a problem that a lot of people here have: not being able to easily search Discord servers. Currentl…