Live data from Hacker News

Magnet-hashes for all torrents on The Pirate Bay: 164 MB

thepiratebay.se

41–50 of 99 posts

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#41
post #27

Earlier quoted context omitted.

68MB with "xz -9 --extreme" :P

70841044 bytes (67.56 MB) by running it through "sort" first. Bonus: it's in chronological order.

it's in sort-of chronological order originally (partially ordered is the correct term, I guess?)

but not 100%

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#42

This gets very meta very quickly. If linking to copyrighted data 'should' be illegal (SOPA), then what about descriptions of that data that are sufficient to identify the original, but not reconstruct it (magnet links)? And if those were made illegal, then what about descriptions of those descriptions? You can recurse infinitely on this. Beyond mere amusement, after just one or two recursions, you get to the point wh…

> This gets very meta very quickly.

You're thinking legality, but I'm thinking efficiency. So we can now distribute 1.5 million torrents (of a total of several thousand TBs, no doubt) in a file 90MB big, in a torrent which itself has a magnet address that takes up...20 bytes?

The size savings as you go up the tree are incredible. I see no reason why you couldn't create an almost-entirely distributed torrent site in this way.

Think about: Torrent discovery could be done by regular distribution of index torrents, and the clients use that to find out what can be downloaded and where.

In fact, in the world of magnet addresses, "uploading" a torrent would be as simple as requesting that its URI be put in the day's index. So running a torrent site would be as simple as curating a list of magnet URIs each day into an index, then publishing that torrent's URI somewhere. Like Twitter. You could run a torrent site entirely from Twitter.

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#43
post #5

Earlier quoted context omitted.

magnet of the magnets: magnet:?xt=urn:btih:938802790a385c49307f34cca4c30f80b03df59c&dn=The+whole+Pirate+Bay+magnet+archive&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80

Should be titled "The whole Pirate Bay magnet archive except this torrent" :-) I think a magnet link is based on the hash of the contents too, so it might be an interesting problem to include the torrent's own magnet link in itself.

The thing is, new torrents are uploaded to Pirate Bay all the time, so one can only archive TPB in any given moment - which has to be, of course, before the creation of this torrent.

The archive is static, the TPB database is dynamic :)

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#44
post #42

This gets very meta very quickly. If linking to copyrighted data 'should' be illegal (SOPA), then what about descriptions of that data that are sufficient to identify the original, but not reconstruct it (magnet links)? And if those were made illegal, then what about descriptions of those descriptions? You can recurse infinitely on this. Beyond mere amusement, after just one or two recursions, you get to the point wh…

> This gets very meta very quickly. You're thinking legality, but I'm thinking efficiency. So we can now distribute 1.5 million torrents (of a total of several thousand TBs, no doubt) in a file 90MB big, in a torrent which itself has a magnet address that takes up...20 bytes? The size savings as you go up the tree are incredible. I see no reason why you couldn't create an almost-entirely distributed torrent site in t…

edonkey has had distributed search for a long time. It's possible to maintain an keyword index of magnet links in a dht, and then you remove the need for the torrent site completely.

I think that requiring torrent files and trackers was a policy decision to deflect liability away from the client implementer to multiple third parties. That's why bit torrent is still around and Grokster isn't. There's no technical need for them.

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#45
post #42

Earlier quoted context omitted.

> This gets very meta very quickly. You're thinking legality, but I'm thinking efficiency. So we can now distribute 1.5 million torrents (of a total of several thousand TBs, no doubt) in a file 90MB big, in a torrent which itself has a magnet address that takes up...20 bytes? The size savings as you go up the tree are incredible. I see no reason why you couldn't create an almost-entirely distributed torrent site in t…

edonkey has had distributed search for a long time. It's possible to maintain an keyword index of magnet links in a dht, and then you remove the need for the torrent site completely. I think that requiring torrent files and trackers was a policy decision to deflect liability away from the client implementer to multiple third parties. That's why bit torrent is still around and Grokster isn't. There's no technical need…

BitTorrent has torrent files and trackers because it was designed for non-infringing use-cases (e.g. distributing Knoppix images). The centralized elements substantially improve reliability (and, at the time, performance) in those use-cases.

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#46

Earlier quoted context omitted.

70841044 bytes (67.56 MB) by running it through "sort" first. Bonus: it's in chronological order.

it's in sort-of chronological order originally (partially ordered is the correct term, I guess?) but not 100%

I don't know how it was ordered originally. I imagine just whatever order the scraper returned data.

putting it throughs sort ordered by tpb's id, which I imagine are assigned in chronological order. The low-numbered torrents seem to be from 2004.

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#47
post #15

Earlier quoted context omitted.

The more one studies computer science, the less one believes that information can be owned in any meaningful way. I remember reading a paper on a "lightnet" that basically XORed arbitrary blocks of data together, then produced a recipe on how to recover some original data by XORing the appropriate blocks. With just this recipe you could ask other nodes for the random blocks, then recover the original data locally. Th…

I'm sure most everyone on HN has read it already, but "What Colour are your bits?" http://ansuz.sooke.bc.ca/entry/23 is always a good reference in this sort of discussion. The system you mention sounds like what the Colour essay is railing against; trying to create a mathematical solution to a problem which is rooted in the law, and cannot be resolved by mathematical manipulations, but only by legal manipulations. Do…

I think parent is talking about Monolith[1], which in fact that article specifically references:

    I think Colour is what the designers of Monolith are trying to challenge, 
    although I'm afraid I think their understanding of the issues is superficial
    on both the legal and computer-science sides. (...)
[1]: http://monolith.sourceforge.net/

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#48
post #32

Earlier quoted context omitted.

Which browser are you using? I am not seeing this no matter how much I resize the window. (I use Nightly, Firefox alpha) Edit: To clarify, the line/link he posted splits across three rows automatically.

Works fine for me on Chrome Stable x64.

It breaks the page on Chrome 16.0.912.77m on Windows 7 x64.

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#49
post #48

Earlier quoted context omitted.

Works fine for me on Chrome Stable x64.

It breaks the page on Chrome 16.0.912.77m on Windows 7 x64.

The stable build is on 17.0.963.46 m. Been a while since you've restart your browser?

Re: Magnet-hashes for all torrents on The Pirate Bay: 164 MB

#50
post #48

Earlier quoted context omitted.

It breaks the page on Chrome 16.0.912.77m on Windows 7 x64.

The stable build is on 17.0.963.46 m. Been a while since you've restart your browser?

Breaks with Chrome 18.0.1017.2.dev on Lucid as well.
Post reply on HN