Live data from Hacker News

Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

news.ycombinator.com

61–70 of 105 posts

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#61

Earlier quoted context omitted.

Thanks for writing that note on the repository -- it was thoughtful and well explained -- reading through your small manifesto that was linked and that's fun too.

I think the manifesto was Aaron Swartz's

oh you're right I didn't even notice -- aaronsw.com... First time I've read it.

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#62
torrent-paradise [0] is a go project which seems to do the same as your project and has existed since 2019. It’s since gone down but remains up on IPFS [1] but its index hasn’t updated since January.

[0] https://github.com/urbanguacamole/torrent-paradise

[1] https://cloudflare-ipfs.com/ipfs/QmQjsKamNFZRvCMXDvZXQmRYjsm...

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#65
You should absolutely open source this. Working with the DHT is a lot of fun, and a great learning exercise.

I adapted my local “torrent roulette” application to an electron app that can be shared. My local version downloads the files, but the one I share only fetches the torrents metadata (easy to adapt it back to my roulette approach). I call it Taboo: https://github.com/retrohacker/taboo

The amount of “evil” on the DHT is pretty low. I’ve run mine for a long time, and very rarely get anything evil. I suspect it’s because of how poorly BitTorrent plays with privacy tools like VPN and Tor, IIUC it’s easy to leak identifying information with BT and its high bandwidth.

Nearly every “evil” file I’ve found is either: a honeypot with not-evil content or password encrypted (maybe to remove the plausible deniability of a random download?). I don’t know if the encrypted files actually contain evil content, I don’t bother trying to crack them and promptly gshred them.

What you will find:

* a lot of content illegal under US copyright law

* a lot of porn (also illegal under US copyright law)

* a metric tonne of fascinating content from other cultures you’d otherwise not be exposed to (also probably illegal under US copyright law)

There is very little “legal” content on the DHT, but most of it is falling on the wrong side of intellectual property law. Sometimes password/credit card dumps. I once found some very sketchy schematics of Eastern European military equipment. But that stuff is also really rare in my experience.

If you do play with these systems, I’ll leave the same warning I left on the Taboo repo:

> Note: I AM NOT A LAWYER! To my knowledge, there aren't any other systems doing this that you can run on your laptop. I suspect the nuance of how Taboo works isn't going to be appreciated by your local law enforcement. If you don't want to test the legality of Taboo in court, I'd strongly recommend either: running a VPN (less safe) or not using Taboo (most safe). If you want to use Taboo with a VPN, may I suggest putting some money in an envelope and sending it to Mullvad?

Adding this on after reading your comments elsewhere in the threads:

If you're worried about Intellectual Property enforcers using this for evil, I wouldn't worry too much about it. BTDigg already exists. Not that what you're doing isn't novel or exciting (great work on this BTW) - but DHT indexing is an art thats already being practiced. Cat is out of the bag.

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#66

I had been working on this successfully for a couple years in the past before I got tired of it and moved on. I still think it's a magnificent idea, to be able to host your own torrent site and to decentralise the last centralised bit of BitTorrent. https://github.com/boramalper/magnetico

I've built a small cli utility to search the database: https://sr.ht/~rakoo/magneticos/. It also asks the DHT and known trackers for the liveness of the swarms, so I can sort by seeders. It's simpler and more useful than running a full webserver with ports and all when I only need an ssh access. It has served me well and will be more than enough for a long time

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#68
post #14

Let it rip. DHT has been around for so long now that whatever bad actors/evil use cases you're imagining have already happened. It sounds like a cool project, and I'd be interested to see it.

I accidentally read this as "Let it R.I.P." I totally agree, unroll it! While the file sharing is distributed, the centralised web-based indexing is still a game of whack-a-mole.

how is what OP made not basically going to be the p1rat3 bay 2.0?

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#69

You should absolutely open source this. Working with the DHT is a lot of fun, and a great learning exercise. I adapted my local “torrent roulette” application to an electron app that can be shared. My local version downloads the files, but the one I share only fetches the torrents metadata (easy to adapt it back to my roulette approach). I call it Taboo: https://github.com/retrohacker/taboo The amount of “evil” on th…

I like how clearly you distinguish "evil" and "not legal" :)

Re: Ask HN: I've Built a DHT Torrent Sniffer and Search Engine. Should I Release?

#70
post #49

Earlier quoted context omitted.

If you follow through to the site linked in the Github: > Tribler is a research project of Delft University of Technology > Work on Tribler has been supported by multiple Internet research European grants. In total we received 3,538,609 Euro in funding for our open source self-organising systems research. Roughly 10 to 15 scientists and engineers work on it full-time. Our ambition is to make darknet technology, secur…

The "robust reputation systems" made me twitch a bit...

See the latest 2022 algorithm called MeritRank [1]. Improves on the prior work developed together with Harvard. We deployed roughly 5 generation in the past 17 years. [1] https://arxiv.org/abs/2207.09950
Post reply on HN