Earlier quoted context omitted.
For me, the hardest part of building a good BT client (or any software, really) is getting the UX right. These days, I use QB after uTorrent's downfall, but even after all these years, its UX still isn’t quite there.
qBittorrent + VueTorrent is not perfect, but it's hard to find anything better. If you haven't tried it, take a look. https://github.com/VueTorrent/VueTorrent
Building a BitTorrent client from the ground up in Go (2020)
41–50 of 93 posts
Re: Building a BitTorrent client from the ground up in Go (2020)
#42Very cool, and surprisingly simple. I was always mystified as a kid how these file-sharing protocols actually work, but if you just look at a torrent as a collection of fixed-size pieces you are asking peers managed by a tracker for, it becomes really straightforward to implement. Now I wonder how clients protect themselves against abusers (i.e.. people who never upload a single bit but only download). I often notice…
IIRC, when both peers only have some subset of pieces, they are meant to “trade” with each other, and so will eventually drop low value peers for higher value ones. Aside from the adversarial case, this helps optimise for being nearer to the people you are sharing with, since torrents are global. In practice, BitTorrent really needs seeders who have downloaded the entirety of the file to be fast for everything except…
For the regular case, 5-10 seeders will hold up an entire torrent indefinitely and without any issues. The contribution leechers make drops off very quickly if good seeders exist.
My algorithm did favour the best peers (both upload and download) but a few years ago switched to a "seeded" ordering to prevent bad behaviour clients dumping or starving new peers.
The garbage uploaders are not an issue. You can isolate peers to pieces and remove them with certainty, or use a technique called smartban which uncovers bad peers very quickly.
Re: Building a BitTorrent client from the ground up in Go (2020)
#43Earlier quoted context omitted.
> Now I wonder how clients protect themselves against abusers (i.e. people who never upload a single bit but only download). It's called leeching, and it depends. It's typically considered good etiquette to upload as much as you download and that's usually enough, but it can be enforced https://en.wikipedia.org/wiki/Leecher_(computing)
I have the opposite problem, many things I download seem so well-seeded (seedboxes?) that I rarely achieve any sort of respectable ratio even if my uplink idles most of the time. It might be client problem but I haven't found much anyone discussing this. TAKE MY BITS!
See my other comment, good seeders carry most of the network these days. Upload bandwidth has become huge since BitTorrent first debuted so true P2P for resource reasons is not really a necessity anymore.
Re: Building a BitTorrent client from the ground up in Go (2020)
#44It is easy to make a torrent client, but very hard to make a good torrent client. A very good, or let's say "perfect", one, needs to support multiple transport protocols (TCP, "uTP" aka UDP, "WebTorrent" aka WebRTC), multiple discovery mechanisms (DHT, PEX, HTTP trackers, WebSocket trackers), multiple torrent formats (v1, v2, hybrid), should use the network optimally (max the speed without overloading the network - I…
Given the state of software quality, it is remarkable that torrent clients are SO GOOD, given they are free!
Re: Building a BitTorrent client from the ground up in Go (2020)
#45It is easy to make a torrent client, but very hard to make a good torrent client. A very good, or let's say "perfect", one, needs to support multiple transport protocols (TCP, "uTP" aka UDP, "WebTorrent" aka WebRTC), multiple discovery mechanisms (DHT, PEX, HTTP trackers, WebSocket trackers), multiple torrent formats (v1, v2, hybrid), should use the network optimally (max the speed without overloading the network - I…
My client anacrolix/torrent has most of that, including streaming, hybrid, v2, all trackers and protocols.
I don't do much port forwarding, it's too fiddly. I think I provide upnp out of the box and few other variants that were contributed.
I also have all the DHT extensions, like mutable torrents and get/put etc.
Large torrents do pop up occasionally, but it's been a long time since someone found a performance issue there but it used to happen every few months. Lots of optimisation.
GitHub.com/anacrolix/torrent
Re: Building a BitTorrent client from the ground up in Go (2020)
#46Earlier quoted context omitted.
> But the "good" news is that nobody built such a "perfect" client just yet, Which of the qualities you listed are lacking from the currently most popular torrent clients (qBitTorrent, etc..)?
I am using qBittorrent, and from the top of my head: - WebTorrent and WebSocket patch for qBittorrent is ready but not merged (waiting on libtorrent's decision), - cross-seeding support is poor (a separate "cross-seed" binary can be used to set up hardlinks to fool qBittorrent into cross-seeding, but it cannot detect duplicates on its own) - when it comes to torrent management, there is no way to group torrents into…
See my other comments, but my client fully supports Webtorrent and webtrackers.
Not much support for port forwarding.
IPv6 definitely works.
Download order is handled by providing readers directly into torrent data and using that for prioritization. So basically request what you need when you need it. No arbitrary list of algorithms.
Re: Building a BitTorrent client from the ground up in Go (2020)
#47It is easy to make a torrent client, but very hard to make a good torrent client. A very good, or let's say "perfect", one, needs to support multiple transport protocols (TCP, "uTP" aka UDP, "WebTorrent" aka WebRTC), multiple discovery mechanisms (DHT, PEX, HTTP trackers, WebSocket trackers), multiple torrent formats (v1, v2, hybrid), should use the network optimally (max the speed without overloading the network - I…
> but very hard to make a good torrent client > So, building a "perfect" torrent client from the ground up is a daunting task. Don’t you think you’re exaggerating a bit? It’s not daunting by any stretch. The feature set you described is fairly straightforward and something even a beginner developer could tackle without too much hassle. I’m honestly kind of tired of seeing people act like doing anything these days is…
Amen.
Re: Building a BitTorrent client from the ground up in Go (2020)
#48Earlier quoted context omitted.
> But the "good" news is that nobody built such a "perfect" client just yet, Which of the qualities you listed are lacking from the currently most popular torrent clients (qBitTorrent, etc..)?
I am using qBittorrent, and from the top of my head: - WebTorrent and WebSocket patch for qBittorrent is ready but not merged (waiting on libtorrent's decision), - cross-seeding support is poor (a separate "cross-seed" binary can be used to set up hardlinks to fool qBittorrent into cross-seeding, but it cannot detect duplicates on its own) - when it comes to torrent management, there is no way to group torrents into…
Re: Building a BitTorrent client from the ground up in Go (2020)
#49One thing that I find hard to understand is how you're able to connect to a random peer online with just an IP and port combination. The peer is likely behind a NAT proxy, and never communicated with you to begin with. In my head, NAT shouldn't allow this connection through, or be able to associate the port with the individual peer machine since the connection source (you) is arbitrary. This is possible with port for…
Most clients try to set up port forwarding using UPnP IGD, NAT PMP, or PCP protocol, a lot of residential routers support one of them. If you enable WebTorrent as a transport protocol (enabled in gotorrent, disabled by default in libtorrent), it should be possible to use existing public STUN/TURN infrastructure, but I don't know if any client does it yet. In practice, you just have to accept that many connections wil…
Yeah I believe WebRTC connections use stun/turn. The feature was contributed and just has a static list somewhere that works fine.
Re: Building a BitTorrent client from the ground up in Go (2020)
#50Very cool, and surprisingly simple. I was always mystified as a kid how these file-sharing protocols actually work, but if you just look at a torrent as a collection of fixed-size pieces you are asking peers managed by a tracker for, it becomes really straightforward to implement. Now I wonder how clients protect themselves against abusers (i.e.. people who never upload a single bit but only download). I often notice…
Sufficiently evolved torrent clients follow a tit-for-tat protocol: - If you're nice with me I'll be nice with you - If you're mean with me I'll be mean with you This is the best answer for the Prisoner's dilemma ( https://en.wikipedia.org/wiki/Prisoner's_dilemma ) on the long run, ie a situation where peers don't trust each other but will both gain if both cooperate. In bittorrent it's typically implemented as follo…
It's only the best answer if the other party is bothering to seek the best answer in real time, and in ways that you can't predict. If your opponent is governed by some algorithm, which is the case for torrent clients, you may need to depart from tit for tat in order to break them out of a needlessly adversarial mode. Or, if you're feeling like the bad guy today, you might need to depart from tit for tat to put them in an irrationally cooperative mode.