Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
1–10 of 111 posts
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#2Very interesting. Does this approach actually work in practice?
Also what happens if illegal content gets scooped up into the index?
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#3Very interesting. Does this approach actually work in practice? Also what happens if illegal content gets scooped up into the index?
Based on my understanding of how the torrent DHT works, all that’s happening is that youre requesting metadata on various torrent info hashes, but that’s not the same thing as actually downloading/seeding the content in the torrent itself.
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#4Very interesting. Does this approach actually work in practice? Also what happens if illegal content gets scooped up into the index?
Considering the screenshot on the linked page shows The Flash among other movies I dont think the author is too concerned about that. I believe its similar to the area that Plex and Jellyfin operate in, mainly that they just provide the framework and tools, what the user does with them is not in their control
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#5Very interesting. Does this approach actually work in practice? Also what happens if illegal content gets scooped up into the index?
It works well in practice. The DHT protocol includes announce messages that broadcast when new files are shared on BitTorrent. It then includes a "geometric" way to find people who are sharing those files. It doesn't include the files themselves, just the torrents which include a file list and location hashes.
If you listen to BitTorrent's DHT network, you'll build an index of everything shared on BitTorrent (over time), this will include commercial movies and such.
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#6Very interesting. Does this approach actually work in practice? Also what happens if illegal content gets scooped up into the index?
It works well in practice. The DHT protocol includes announce messages that broadcast when new files are shared on BitTorrent. It then includes a "geometric" way to find people who are sharing those files. It doesn't include the files themselves, just the torrents which include a file list and location hashes. If you listen to BitTorrent's DHT network, you'll build an index of everything shared on BitTorrent (over ti…
>It works well in practice.
Hi, I worked on gnutella and lots of P2P systems in the early 00s. This will devolve into noise and spam as the number of users who adopt this feature pass a critical mass. With a fully decentralized system, there are no gatekeepers, and as such, there is no way to filter counterfiet items. While your client will present with you the data you are searching for, you will find out (usually hours later) that your supposed pirated download is actually just a 2hour loop of Rick Astley (still piracy though, so you are still winning.. i think?).
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#7Earlier quoted context omitted.
It works well in practice. The DHT protocol includes announce messages that broadcast when new files are shared on BitTorrent. It then includes a "geometric" way to find people who are sharing those files. It doesn't include the files themselves, just the torrents which include a file list and location hashes. If you listen to BitTorrent's DHT network, you'll build an index of everything shared on BitTorrent (over ti…
>It works well in practice. Hi, I worked on gnutella and lots of P2P systems in the early 00s. This will devolve into noise and spam as the number of users who adopt this feature pass a critical mass. With a fully decentralized system, there are no gatekeepers, and as such, there is no way to filter counterfiet items. While your client will present with you the data you are searching for, you will find out (usually h…
But you can still pick the option with the most seeders, which should get you what you're looking for most of the time.
The spam problem isn't nonexistent within the centralized services either.
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#8Earlier quoted context omitted.
It works well in practice. The DHT protocol includes announce messages that broadcast when new files are shared on BitTorrent. It then includes a "geometric" way to find people who are sharing those files. It doesn't include the files themselves, just the torrents which include a file list and location hashes. If you listen to BitTorrent's DHT network, you'll build an index of everything shared on BitTorrent (over ti…
>It works well in practice. Hi, I worked on gnutella and lots of P2P systems in the early 00s. This will devolve into noise and spam as the number of users who adopt this feature pass a critical mass. With a fully decentralized system, there are no gatekeepers, and as such, there is no way to filter counterfiet items. While your client will present with you the data you are searching for, you will find out (usually h…
The way to filter out garbage is to download things with lots of seeds, and if you still happen to download garbage, to immediately stop sharing it.
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#9It'd be so nice to have a super simple DHT crawler CLI tool, in both implementation and interface.
Re: Bitmagnet: A self-hosted BitTorrent indexer, DHT crawler, and torrent search
#10Earlier quoted context omitted.
It works well in practice. The DHT protocol includes announce messages that broadcast when new files are shared on BitTorrent. It then includes a "geometric" way to find people who are sharing those files. It doesn't include the files themselves, just the torrents which include a file list and location hashes. If you listen to BitTorrent's DHT network, you'll build an index of everything shared on BitTorrent (over ti…
>It works well in practice. Hi, I worked on gnutella and lots of P2P systems in the early 00s. This will devolve into noise and spam as the number of users who adopt this feature pass a critical mass. With a fully decentralized system, there are no gatekeepers, and as such, there is no way to filter counterfiet items. While your client will present with you the data you are searching for, you will find out (usually h…
While you're technically correct, the protocol is resilient to such attack, as the number of people participating in a particular torrent is a good indicator of its validity. After all, everyone who was fooled will delete and stop sharing such items.
New releases of something that just came out tend to suffer from this, though. Sometimes the counterfeits reach escape velocity - the rate of people joining in downloading the counterfeit exceed the rate of people realizing and stopping, thus giving the illusion of a legit torrent.
Currently this problem is being solved by torrent sites' reputation and comment systems. If we imagine a world where only decentralized indexes like Bitmagnet exist, your prediction is 100% accurate. This only works if reputation from a reliable site is bootstrapping the initial popularity of a torrent.