Earlier quoted context omitted.
I've had one running for over a year now, it's replaced my usage of regular torrent sites completely, there is a lot of junk, and it gets stale, but it's still a better experience than most of the public trackers out there IMO
How much space do you need to store the index?
Crawling BitTorrent DHTs for Fun and Profit [pdf]
41–50 of 52 posts
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#42Bitmagnet - https://bitmagnet.io/ - does exactly that. I left it running for a few weeks and then stopped the crawler. Didn't expect much, but still somewhat disappointed by the garbage it reeled in.
why the hell does that require a postgres database instance? has no one heard of sqlite? "It's just `docker compose up` dude." that's not excuse for unnecessary complexity. software for a single person should be a single binary that anyone can run how they choose. If they want to use Docker, great. If they want to run it in a terminal, great. If they want to run it on a server, great. If they want to run it on their…
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#43Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#44The article neglects to define "DHT" before using it.
No it doesn't. Before you try to correct me, the abstract never counts ;)
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#45Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#46The article neglects to define "DHT" before using it.
Distributed hash table - ButTorrent extension for discovering torrent's seeders by advertising its hash across known peer pool, think of it as a distributed tracker. Contrary to traditional way of asking a known tracker for peers of that torrent. Its algorithm is very elegant, using binary search on peers' and torrents' hashes, narrowing down to peers that are more likely to be seeders (or at least know some). https:…
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#47The article neglects to define "DHT" before using it.
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#48The article neglects to define "DHT" before using it.
Downvote all you want, but a fact is a fact: The article doesn't define the abbreviation before using it.
One search tells you what DHT in the context of wikipedia is: https://en.wikipedia.org/wiki/Mainline_DHT
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#49Earlier quoted context omitted.
I'm starting to upload everything very soon. I have >4million magazines here so far I think. Feel free to email me on my profile :)
Do you have old 2000s era Argos catalogues? Argos is a shop in the UK, I've been tracking down old catalogues to see if my childhood CD player is in there, can't find evidence of it anywhere and so many were made. The catalogues are sort of rare but were given away for free, now people sell old ones on ebay
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#50Earlier quoted context omitted.
probably worth adding some ML filter to it because yeah, most of the bulky stuff in bittorrent is always going to be garbage - a lot like the internet generally, the value is in filtering the good stuff out
out of curiosity, what kind of junk/garbage is typical?
quality and traffic are grossly uncorrelated, so finding stuff is a huge challenge