Live data from Hacker News

Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

torrentfreak.com

91–100 of 103 posts

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#91

Earlier quoted context omitted.

Try https://www.coveapp.info/ , I mentioned it here: https://news.ycombinator.com/item?id=39425381 . I wrote a custom search implementation for torrents that uses keyword matching and boosts search scores based on popularity of torrents on the DHT. It feels very much like TPB search but not so aggressively focused on seeder counts (i.e. better matching on file names can outrank a higher seeder count).

Why is there an ffmpeg dependency if the app is only working with metadata?

It uses ffprobe for metadata, and ffmpeg for transcoding.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#92

I'm the author of https://github.com/anacrolix/torrent (started in 2013) and https://github.com/anacrolix/dht (started in 2015). I have a DHT indexer implementation I developed in 2021. It's currently closed source but available for use as part of https://www.coveapp.info/ . I have found that after several hours the search is excellent and stays up to date with ease.

Some of the features in your screenshots look interesting, but why is Cove closed source?

Mainly because I am not ready to share my DHT indexing implementation. It's the culmination of years of working with BitTorrent and DHTs and I'd like to get something back for it one day. However I do dogfood Cove and want people to have that experience too.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#93
post #43

Earlier quoted context omitted.

> You have to be participating in a torrent swarm in order to bootstrap yourself into DHT at all. This is not correct. > Bittorrent's DHT is not a network independent of torrent swarms. This is also wrong. People have used the DHT for non-torrent-related purposes. > You need the address of a peer who is already part of the network in order to join it, and you have to get a list those addresses from somewhere. And not…

Which of these methods is not obtaining the address of a node in a swarm, or hitting a tracker for a list of nodes in a swarm? > People have used the DHT for non-torrent-related purposes. This is simply non-responsive. People have used a DHT overlaid on the collection of torrent swarms for non-torrent related purposes. > And nothing dictates that that has to be obtained via the bittorrent peer protocol. This is a sil…

Sorry bro, the8472 is correct on all counts. Misunderstanding DHTs and BitTorrent and how they relate is very common.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#94
post #90

Implementing sample_infohashes opens your torrent client to abuse. UDP responses are much larger than queries and this protocol (BEP51) allows attackers that can spoof source IP addresses to use a large number of clients as mules for amplified distributed denial of service attacks.

Individual DHT nodes should only see a trickle of packets. A few kilobytes per second. Even less per remote IP. So they can set fairly strict rate limits. And ISPs are in a much better position to solve this problem anyway. They should ask for higher peering fees from peers that don't do source-filtering.

True. Although an ISP can say they do source filtering and then fail to implement it properly. For example my ISP at home implements source filtering on IPv4, but not on IPv6.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#95
post #67

Earlier quoted context omitted.

Then you have a user reputation problem instead… me and my 5 million alts say this is the REAL korn_blind.mp3.exe. Nothing is new under the sun since the internet was created, these are the same problems that email has tried and failed to solve for 50 years.

Its going to be an unpopular take, but crypto solves this. As in currency. Make people attest to things with locking a very small amount of money.

Bitcoin originally evolved from the "hashcash" proof-of-work system, which was intended to be a scalable anti-spam measure (force an attacker to generate a unique proof that is expensive to generate but cheap to verify). And you are basically describing proof-of-stake (force participants to stake, and if they misbehave you slash their stake). I wasn't kidding about "nothing is new under the sun" ;)

In general though others are right that the problem often is that attackers are more willing to play these games than individuals - you are not willing to put up 10 bucks to post on a web forum, but an attacker might think $10 to get a wonderful commercial offer delivered to 1000 people sounds really good! You won't spend 10c to upvote each individual song, but the RIAA will! Etc. It needs to be highly asymmetrical, and ideally have minimal/zero cost to "honest" users with an exponential penalty for attackers.

Obviously we are not living in a post-email-spam world unfortunately, and what we have is basically the "lightning network" with gatekeepers offloading and centralizing a lot of the problem so users don't have to deal with it. But you aren't the first person to make the observation that these are related problems!

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#96
post #49

"Unlike well-moderated torrent sites, Bitmagnet adds almost any torrent it finds to its database. This includes mislabeled files, malware-ridden releases, and potentially illegal content. The software tries to limit abuse by filtering metadata for CSAM content, however." This is by far the biggest hurdle to something like this. You'll eventually have to end up again with a centralized curator.

All the CSAM torrents are fed honeypots. Just don't accidentally download them and you should be fine.

How do you know? And why would you know?

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#97
Isn't this dangerous? I thought that in order to get details of any torrent from the DHT you must connect to it. This would automatically set off DMCA complaints from MPAA/RIAA etc. right?

(also how do they ever prove you actually downloaded any usable part of a torrent?!)

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#98

Isn't this dangerous? I thought that in order to get details of any torrent from the DHT you must connect to it. This would automatically set off DMCA complaints from MPAA/RIAA etc. right? (also how do they ever prove you actually downloaded any usable part of a torrent?!)

You must connect to peers yes, but metadata is separate to torrent data.

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#99

Isn't this dangerous? I thought that in order to get details of any torrent from the DHT you must connect to it. This would automatically set off DMCA complaints from MPAA/RIAA etc. right? (also how do they ever prove you actually downloaded any usable part of a torrent?!)

You must connect to peers yes, but metadata is separate to torrent data.

but copyright holders do not need to show you actually DID anything besides connect to the torrent in order to accuse you, that's all they ever do...

Re: Bitmagnet Allows People to Run Their Own Decentralized Torrent Indexer Locally

#100

Earlier quoted context omitted.

Some of the features in your screenshots look interesting, but why is Cove closed source?

Mainly because I am not ready to share my DHT indexing implementation. It's the culmination of years of working with BitTorrent and DHTs and I'd like to get something back for it one day. However I do dogfood Cove and want people to have that experience too.

I would think the only thing you would get out of such work is a sternly worded cease and desist from a big entity so you might as well open-source this and let somebody else take the torch in case you are shut down.

I mean I'd use such software (and huge DBs like that of Skytorrentz) for research purposes because to me distribution is hugely interesting as a hobby but many courts won't see it that way, and we know a lot of them are influenced by copyrights holders.

Post reply on HN