Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

torrentfreak.com

Post reply on HN
Print view View original

31–40 of 103 posts

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#31

Post by pluto_modadic »

the8472 wrote: post #13
KoftaBob wrote: post #5

Earlier quoted context omitted.

Well for your use case of distributing your own music, Bitmagnet wouldn't be necessary. What a DHT crawler like Bitmagnet does is the following: 1. Take a few initial "bootstrap" torrents and ping them to see which IP addresses are seeding that file 2. Ping those IP addresses and ask what other files/torrents they're seeding 3. Ping those torrents to see which IP addresses are seeding that torrent Rinse repeat. So to…

That's not correct. Torrent swarms and the DHT are separate. Each torrent basically forms its own small network of TCP connections to exchange data specific to that one torrent. While the DHT is a network shared by all clients that speak the protocol and it's carried over short-lived UDP query-response exchanges.

they're saying they jump between one and the other. ask DHT, then ask swarm, then ask DHT, then ask swarm.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#32

Post by pluto_modadic »

gkbrk wrote: post #12
KoftaBob wrote: post #5

Earlier quoted context omitted.

Well for your use case of distributing your own music, Bitmagnet wouldn't be necessary. What a DHT crawler like Bitmagnet does is the following: 1. Take a few initial "bootstrap" torrents and ping them to see which IP addresses are seeding that file 2. Ping those IP addresses and ask what other files/torrents they're seeding 3. Ping those torrents to see which IP addresses are seeding that torrent Rinse repeat. So to…

You don't need to ping anyone to crawl the DHT. You can passively wait and you'll get DHT queries in the form of "I'm looking for people seeding XYZ. Do you have a list?". You can just save those somewhere and you'll accumulate a list of active and new torrents. Writing a DHT crawler is super fun, I suggest everyone to get a cheap VM and write/run one.

that'd be the slow mode, though

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#33

Post by mgdigital »

pluto_modadic wrote: post #30
mgdigital wrote: post #28

Earlier quoted context omitted.

Bitmagnet has rate limiting on incoming UDP requests (both overall and per-IP) so I don't know that it would be vulnerable; if there's anything else that should be done to mitigate any risk I'd like to know.

outgoing rate limits, and also it could exhaust the overall rate with useless traffic.

Incoming and outgoing are both limited, I think the worst such an attack could do is prevent responding to legitimate incoming queries - this shouldn't slow down the DHT crawler in a noticeable way.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#34

Post by jMyles »

gkbrk wrote: post #12
KoftaBob wrote: post #5

Earlier quoted context omitted.

Well for your use case of distributing your own music, Bitmagnet wouldn't be necessary. What a DHT crawler like Bitmagnet does is the following: 1. Take a few initial "bootstrap" torrents and ping them to see which IP addresses are seeding that file 2. Ping those IP addresses and ask what other files/torrents they're seeding 3. Ping those torrents to see which IP addresses are seeding that torrent Rinse repeat. So to…

You don't need to ping anyone to crawl the DHT. You can passively wait and you'll get DHT queries in the form of "I'm looking for people seeding XYZ. Do you have a list?". You can just save those somewhere and you'll accumulate a list of active and new torrents. Writing a DHT crawler is super fun, I suggest everyone to get a cheap VM and write/run one.

It's kinda like the owl with the tootsie pop:

How many cups of coffee does it take to internalize hamming distance? :-)

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#35

Post by jMyles »

8organicbits wrote: post #3
jMyles wrote: post #2

Interesting. I want to seamlessly distribute my music through bittorrent, but since I have a small fan base (and thus, a very small potential seeding pool), I found it difficult to connect all the moving parts. I'll give Bitmagnet a try for an Indexer. I did find that IPFS, with a pinning service (I won't shill the one I used in particular), was a bit easier and just worked for everyone who tried to use it. But I'd l…

Have you considered setting up a web seed? https://www.ubuntubuzz.com/2021/06/how-to-add-web-seeds-to-t...

No I hadn't!

This seems exciting. Have you done it successfully?

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#36

Post by jMyles »

rakoo wrote: post #8
jMyles wrote: post #2

Interesting. I want to seamlessly distribute my music through bittorrent, but since I have a small fan base (and thus, a very small potential seeding pool), I found it difficult to connect all the moving parts. I'll give Bitmagnet a try for an Indexer. I did find that IPFS, with a pinning service (I won't shill the one I used in particular), was a bit easier and just worked for everyone who tried to use it. But I'd l…

You don't need an indexer for that. All you need is a static list with all your music, and people can click on links to download your music. Give them qbittorrent and it'll be running smoothly.

Sure, but I'd like to go beyond magnet links and make it properly decentralized, with the discovery happening in the DHT and torrent swarm.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#38

Post by rakoo »

toyg wrote: post #24
Scion9066 wrote: post #20

Earlier quoted context omitted.

In order to get the information such as the name of the torrent and its files from the hash you do need to connect to someone in the swarm to download that metadata. You won't know what it is until after you've already connected.

Connecting to an unknown machine and asking what they have, is like knocking on a stranger's door and asking what they're selling. Them mentioning something nefarious and you leaving in response, is very obviously not a crime.

There probably are nefarious content you can see just from the filenames but not everything is like that. Moreover, you "only" know they distribute it, you don't do it yourself.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#39

Post by [deleted] »

bangonkeyboard wrote: post #17

Who remembers Skytorrents ( https://news.ycombinator.com/item?id=13423629 )? Posted as a "Show HN" here, it was a DHT-sourced index and stack written in C with no JavaScript, no cookies, no ads, no tracking. Skytorrents was unbelievably fast, friendly, and complete, and this translated into rapid adoption and traffic growth that caused the site to shut down due to server costs after just a year ( https://torrentfreak…

[deleted]
Post reply on HN