Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Crawling BitTorrent DHTs for Fun and Profit [pdf]

usenix.org

Post reply on HN
Print view View original

11–20 of 52 posts

Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]

#14

Post by ivanjermakov »

MoonWalk wrote: post #12

The article neglects to define "DHT" before using it.

Distributed hash table - ButTorrent extension for discovering torrent's seeders by advertising its hash across known peer pool, think of it as a distributed tracker. Contrary to traditional way of asking a known tracker for peers of that torrent.

Its algorithm is very elegant, using binary search on peers' and torrents' hashes, narrowing down to peers that are more likely to be seeders (or at least know some).

https://www.bittorrent.org/beps/bep_0005.html

Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]

#16

Post by qingcharles »

NoMoreNicksLeft wrote: post #10
qingcharles wrote: post #6

Earlier quoted context omitted.

Are you running it at home? I built one with a nice TUI to run on a VPS so I can try and find rare magazine torrents, but Hetzner were upset about it. I need to find it a new home. It was a very good citizen, but it still raised too many flags.

Which magazines?

Anything I don't have! Sometimes I'll find a torrent and no seeds/peers and I'll wonder if there is another torrent out there that has the same files in it somewhere that I can find.

The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.

Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]

#17

Post by qingcharles »

hdgr wrote: post #3

Bitmagnet - https://bitmagnet.io/ - does exactly that. I left it running for a few weeks and then stopped the crawler. Didn't expect much, but still somewhat disappointed by the garbage it reeled in.

This is a good tip, thanks. I'll probably replace my home-grown scanner for this one.

Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]

#19

Post by loeg »

ivanjermakov wrote: post #14
MoonWalk wrote: post #12

The article neglects to define "DHT" before using it.

Distributed hash table - ButTorrent extension for discovering torrent's seeders by advertising its hash across known peer pool, think of it as a distributed tracker. Contrary to traditional way of asking a known tracker for peers of that torrent. Its algorithm is very elegant, using binary search on peers' and torrents' hashes, narrowing down to peers that are more likely to be seeders (or at least know some). https:…

Not a P2P innovation with Bittorrent, FWIW. Kademlia DHT (used in eMule/LimeWire/Gnutella P2P networks) long predates Bittorrent.

Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]

#20

Post by toomuchtodo »

qingcharles wrote: post #16
NoMoreNicksLeft wrote: post #10

Earlier quoted context omitted.

Which magazines?

Anything I don't have! Sometimes I'll find a torrent and no seeds/peers and I'll wonder if there is another torrent out there that has the same files in it somewhere that I can find. The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.

Can I get a copy for the Internet Archive? Will take as much of the corpus as you’re willing to provide.

(no affiliation with them)

Post reply on HN