Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updating Torrents via DHT Mutable Items

bittorrent.org

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Updating Torrents via DHT Mutable Items

#31

Post by aninhumer »

Mizza wrote: post #2

This is an okay idea, but it doesn't really solve the big problem facing BT right now: content discovery. Mutable Items are good for updating content, but that doesn't matter if nobody can find it. There needs to be a BEP for a way to host, serve and search metadata about torrents, not just their info-hashes. This should be priority #1 for BT devs. Fortunately, there is at least some related thinking in this directio…

I remember someone setting up a bittorrent search engine based on DHT scraping. It's probably been taken down, but I think it was called DHT Dig, and there's apparently a library with that name.

Something like that would make it easy to create centralised search engines for casual users, and allow power users to run local databases.

Re: Updating Torrents via DHT Mutable Items

#32

Post by jokoon »

pessimizer wrote: post #22
DarkLinkXXXX wrote: post #21

Earlier quoted context omitted.

To be fair, torrents were always a fantastic vector for malware.

To be fair, the internet has always been a fantastic vector for malware.

To be fair, human cells has always been a fantastic vector for infection.

Re: Updating Torrents via DHT Mutable Items

#33

Post by beefield »

niftich wrote: post #27
beefield wrote: post #26

So, what is stopping this to be the ultimate distributed social media platform? I mean, I should be able to publish a relatively simple sqlite database that contains: - my public posts - my public encryption key - list of my friends - my private posts encrypted with the friends' public keys Then I should seed at least my and my friends databases, so even if some of my friend is offline, his/hers database is still ava…

Network effects and lack of off-the-shelf implementations.

Network effects are clearly a major issue, but wrapping the concept to a webpage and/or mobile app should not be a major hurdle. If I have not missed something.

If I only had some more time on my hands...

Re: Updating Torrents via DHT Mutable Items

#34

Post by xorcist »

cocotino wrote: post #5
sktrdie wrote: post #4

Earlier quoted context omitted.

Torrent sites could publish their database dumps via this extension, and consumers would automatically download their updated index. A torrent can be anything; it can also be a list of other torrents ;) So it does help with content discovery.

An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have p2p search?

Kademlia search is also rather useless as it is flooded with false results. This is of interest not only to malware writers but also as a service to sell to rights holders.

Re: Updating Torrents via DHT Mutable Items

#35

Post by sktrdie »

Mizza wrote: post #7
sktrdie wrote: post #4

Earlier quoted context omitted.

Torrent sites could publish their database dumps via this extension, and consumers would automatically download their updated index. A torrent can be anything; it can also be a list of other torrents ;) So it does help with content discovery.

You don't see a problem with that..? You still need to have the entry point - that's exactly where the censorship is occurring. Try to find a copy of the KickAssTorrents database on Google right now and tell me how easy it is. The approach you're talking about is essentially what http://bitcannon.io/ is trying to do - but that requires running a separate app, a local MongoDB server, downloading many gigabytes of data…

If KickAss decides to share their dump using their public key, it becomes much harder to censor because of how the DHT system works. Authorities would need to shut down all the nodes that republish that public key on the DHT, which is incredibly hard to do. Much harder than asking ISPs to shutdown a domain name in DNS.

Re: Updating Torrents via DHT Mutable Items

#36

Post by brownhats »

niftich wrote: post #14
Mizza wrote: post #13

Earlier quoted context omitted.

Why is that fair? They're designed for different purposes. IPFS falls far short of being useful in a transformative way because it lacks this important feature. Otherwise, it'll be a kind of gimmicky alternative to HTTP, "auto-archiving HTTP", that never got used for anything real.

I don't disagree that having content discovery would be nice, but IFPS makes a pretty good case [1] about what it brings to the table over HTTP. My point was that HTTP also has "zero out-of-the-box support" for content discovery, yet once search engines came out, we were fine. [1] https://ipfs.io/#why

If by fine you mean a centralized search engine is a good solution for a decentralized network, then yeah.

Re: Updating Torrents via DHT Mutable Items

#37

Post by Senji »

pepijndevos wrote: post #29
eximius wrote: post #18

Wow, the first thing I think of is that this could be a fantastic vector for malware. Anyone who implements this should make it very clear the underlying data can change.

At least now you can hope that popular torrents with positive comments are maybe okay. With this the author could simply add the malware after it has become popular.

Not a problem. The tracker will mark the torrent as changed from its previous version.

Re: Updating Torrents via DHT Mutable Items

#38

Post by rakoo »

eximius wrote: post #18

Wow, the first thing I think of is that this could be a fantastic vector for malware. Anyone who implements this should make it very clear the underlying data can change.

You don't stumble randomly on such a mutable torrent. If you know git's data model, usual torrents are like commits, and those mutable torrents are like branches: If you give someone a commit there is trust that it won't move, and if you give them a branch name, there is expectation that the actual commit pointed to will change.

Re: Updating Torrents via DHT Mutable Items

#39

Post by rakoo »

aninhumer wrote: post #31
Mizza wrote: post #2

This is an okay idea, but it doesn't really solve the big problem facing BT right now: content discovery. Mutable Items are good for updating content, but that doesn't matter if nobody can find it. There needs to be a BEP for a way to host, serve and search metadata about torrents, not just their info-hashes. This should be priority #1 for BT devs. Fortunately, there is at least some related thinking in this directio…

I remember someone setting up a bittorrent search engine based on DHT scraping. It's probably been taken down, but I think it was called DHT Dig, and there's apparently a library with that name. Something like that would make it easy to create centralised search engines for casual users, and allow power users to run local databases.

There was btdigg at some point (https://btdigg.org/) but for some reason it's down today. The approach is not particularly complicated, there even were papers about that:

* In Vuze: https://www.usenix.org/legacy/event/woot10/tech/full_papers/..., slides at https://www.defcon.org/images/defcon-18/dc-18-presentations/...

* In Mainline: https://arxiv.org/ftp/arxiv/papers/1009/1009.3681.pdf

Basically all you have to do is sit there, people will come to you asking for content, you'll ask as a standard DHT node but note that the infohash exists, and get more information (name, number of peers, ...) about it in the background. All you need is a few IPs.

Re: Updating Torrents via DHT Mutable Items

#40

Post by rakoo »

aninhumer wrote: post #30
niftich wrote: post #3

This is an intriguing addition to Bittorrent; this kind of feature exists in IPFS [1], but of course Bittorrent has a much, much larger install base, so this progressive enhancement could bring similar or near-equivalent features to a wider audience. This implementation relies on BEP 44 (Storing arbitrary data in the DHT) [2], which, together with the design of the DHT, does have some security implications [3]. [1] h…

Indeed, I'm surprised it's taken this long for people to take advantage of bittorrent DHT for other p2p stuff. You don't even need to change the protocol to start using it. It's a ready made hash-to-peerlist mapping, so you could just use it for peer-discovery/bootstrapping and do other stuff out-of-band.

There already was a BEP to exchange arbitrary stuff, BEP44, for mutable or immutable items. It's taken this long to use this BEP specifically for mutable torrents, though, because it turned out that not many people had a need for mutable torrents.
Post reply on HN