Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
1–10 of 35 posts
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#2I like the quick fix the site designer used to switch from a static layout to a fluid one.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#3[deleted]
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#4Interesting, but somehow doubt that many people have the set-up to handle this number of data.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#5Doesn't include deleted articles, so no hope if you want to recover one of them. This is a pity since Wikipedia deletes too many articles.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#6The article says Tb not TB, but in reality it appears to be TB. That's quite a difference. Still seems heavy for text, but I assume the full text of every revision is in it, not just diffs.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#740 + 15 days to compress? how long it would take to decompress this thing
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#840 + 15 days to compress? how long it would take to decompress this thing
I imagine that the compression is only for saving time during transfer.
That, and hopefully you could decompress just what you needed. I.E. a tarball of compressed articles.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#9Interesting, but somehow doubt that many people have the set-up to handle this number of data.
That's because, despite what the 14 year olds on digg and reddit think, this isn't for you to download on your computer at your house. This is for archival or data-mining purposes.
I apologize for the minor insult at digg/reddit, I just remember a few years ago a link to the archive was posted on digg and everyone started downloading it...unnecessarily wasting wikipedia's limited and donated resources.
Re: Full-history English Wikipedia dump produced: 5.6TB uncompressed, 32GB 7z'd
#1040 + 15 days to compress? how long it would take to decompress this thing
7zip, like many other compression schemes, is optimized and designed so that decompression is typically (much) faster than compression.
The web page (http://www.7-zip.org/7z.html) states that the default "native" LZMA format decompresses at between 10 and 20 times the speed that it compresses.
So, 15 days / 15 is about one day to decompress, then.