Earlier quoted context omitted.
I've had one running for over a year now, it's replaced my usage of regular torrent sites completely, there is a lot of junk, and it gets stale, but it's still a better experience than most of the public trackers out there IMO
Same here, for over a year. how many torrents has yours indexed?
Crawling BitTorrent DHTs for Fun and Profit [pdf]
31–40 of 52 posts
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#32Earlier quoted context omitted.
probably worth adding some ML filter to it because yeah, most of the bulky stuff in bittorrent is always going to be garbage - a lot like the internet generally, the value is in filtering the good stuff out
out of curiosity, what kind of junk/garbage is typical?
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#33Earlier quoted context omitted.
Anything I don't have! Sometimes I'll find a torrent and no seeds/peers and I'll wonder if there is another torrent out there that has the same files in it somewhere that I can find. The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.
Can I get a copy for the Internet Archive? Will take as much of the corpus as you’re willing to provide. (no affiliation with them)
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#34Earlier quoted context omitted.
Anything I don't have! Sometimes I'll find a torrent and no seeds/peers and I'll wonder if there is another torrent out there that has the same files in it somewhere that I can find. The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.
High Times is mostly on archive.org, if you need that one. I'd sort of like the film-making one, I'll put some time into that. On my list of periodicals, I think the count's up to 500 that I consider important enough to archive and I'm nowhere near done with it.
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#35Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#36Earlier quoted context omitted.
I've had one running for over a year now, it's replaced my usage of regular torrent sites completely, there is a lot of junk, and it gets stale, but it's still a better experience than most of the public trackers out there IMO
I disabled mine because it was constantly writing to my SSD.
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#37You would need a unique IP to overcome this per n nodes. That could be tens of thousands of IPs.
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#38Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#39Bitmagnet - https://bitmagnet.io/ - does exactly that. I left it running for a few weeks and then stopped the crawler. Didn't expect much, but still somewhat disappointed by the garbage it reeled in.
"It's just `docker compose up` dude." that's not excuse for unnecessary complexity. software for a single person should be a single binary that anyone can run how they choose. If they want to use Docker, great. If they want to run it in a terminal, great. If they want to run it on a server, great. If they want to run it on their phone, great.
developers: avoid limiting people artificially like this. There is no way that Sqlite is insufficient for this.
Re: Crawling BitTorrent DHTs for Fun and Profit [pdf]
#40Earlier quoted context omitted.
Can I get a copy for the Internet Archive? Will take as much of the corpus as you’re willing to provide. (no affiliation with them)
I'm starting to upload everything very soon. I have >4million magazines here so far I think. Feel free to email me on my profile :)