Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

7-Zip for Windows can now use more than 64 CPU threads for compression

7-zip.org

Post reply on HN
Print view View original

201–210 of 220 posts

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#201

Post by birksherty »

Beretta_Vexee wrote: post #48
d33 wrote: post #10

I worry that 7-Zip is going to lose relevance because lack of zstd support. zlib's performance is intolerable for large files and zlib-ng's SIMD implementation only helps here a bit. Which is a shame, because 7-Zip is a pretty amazing container format, especially with its encryption and file splitting capabilities.

You are looking for 7-Zip Zstd: https://github.com/mcmilk/7-Zip-Zstd I don't know what your use case is, but it seems to be quite a niche.

I use this one too. This is the best one out of all the alternative 7zip with zstd. UI is same, has all the options and faster than those for zstd in my comparisons.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#202

Post by soruly »

Szpadel wrote: post #196
soruly wrote: post #122

Earlier quoted context omitted.

I think it depends on what you're compressing. I experimented with my data full of hex text xml files. xz -6 is both faster and smaller than zstd -19 by about 10%. For my data, xz -2 and zstd -17 achieve the same compressed size but xz -2 is 3 times faster than zstd -17. I still use xz for archive because I rarely needs to decompress them.

Try combining it with --long My use cases are usually source code, SQL dumps and log files. Sometimes xz gave marginally better results, but difference was well below 1%

thanks for the tips. As my data has very low entropy, both can compress down to 3-4% of original size, but xz is a lot faster in compression.

raw size: 9612344 B

zstd --ultra -22 --long=31 => 376181 B (3.91% original, 4.088s compress, 0.013s decompress)

xz -z -9 xml => 353700 B (3.68% original, 0.729s compress, 0.032s decompress)

zstd -17 --long=31 could match the compression time of xz, but the size is bigger (405602 B, 4.22% original)

If you compare only the compressed size (not to the original size), .zst would be about 6-15% larger than .xz

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#203

Post by ltbarcly3 »

tobinc wrote: post #135
ltbarcly3 wrote: post #22

Wow, a program that doesn't matter anymore has been very very minimally enhanced on a platform that doesn't matter anymore, benefitting the 7 users that have more than 64 real cores with Windoes and are regularly compressing archives so large that it doesn't drastically reduce the compression ratio to split it into more thsn 64 sections. Posting this link to hn has consumed more human potential than the thing it is d…

A 1% speed improvement for 1% of 7zip users is several times more productive than your comment.

This will not deliver your stated threshold.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#204

Post by ltbarcly3 »

starkrights wrote: post #152
ltbarcly3 wrote: post #22

Wow, a program that doesn't matter anymore has been very very minimally enhanced on a platform that doesn't matter anymore, benefitting the 7 users that have more than 64 real cores with Windoes and are regularly compressing archives so large that it doesn't drastically reduce the compression ratio to split it into more thsn 64 sections. Posting this link to hn has consumed more human potential than the thing it is d…

> a program that doesn’t matter anymore The rest of this comment has, though gratuitously snarky, a point, but I don’t think claiming that 7zip is irrelevant as an independent statement is even remotely coherent.

tar.zstd is superior in basically every way, open, and portable

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#205

Post by redeeman »

GuB-42 wrote: post #185
redeeman wrote: post #177

Earlier quoted context omitted.

im okay with that. That being said, I have not had a single issue delivering zip files with lzma, and i KNOW that I have gotten MANY from the random sources. I would also expect people to be able to decode h265 in an mp4 file. Your proposal seems, to word it bluntly, retarded. You would have mp4 frozen for h264 for ETERNITY, and then invent a new format as replacement? or you would just say "god has bestowed upon the…

It depends on your priorities. If your customer is stuck in the 90s because his 90s technology works perfectly fine and he has no intention to fix things that are not broken. Then deliver stuff that is compatible with 90s technology. He will be happy, will continue to work with you and you will make money. If your customer is using the latest technologies and values size efficiency, then use the latest codecs. I usua…

not everything is a clientcustomer relationship.

Professionally I can definitely support old stuff. It costs extra most often.

Conservative doesnt have to be stuck. I am not recommending we send h266 to everyone now, but h265 is well supported, as is AV1.

lzma support in zip has been widely supported for many years at this point. I am going to be choosing my "sane defaults", and if someone has a problem with that, they can simply do what they need to do to open it, or provide a damn good reason for me to go out of my way.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#206

Post by jorvi »

sidewndr46 wrote: post #70
jorvi wrote: post #25

Earlier quoted context omitted.

.. but 7-zip has a pretty terrible GUI? Hence why PeaZip is so popular, and J-Zip used to be before it was stuffed with adware.

If you're expecting a "mobile first" or similar GUI where most of the screen is dedicated to whitespace, basic features involves 7 or more mouse clicks and for some reason it all gets changed every ~6 months then yes the 7zip GUI is terrible. Desktop software usability peaked sometime in the late 90s, early 2000s. There's a reason why 7zip still looks like ~2004

You could have taken the 10 seconds to type in "PeaZip GUI" and seen that it is not a mobile interface and it is indeed much nicer than the 7-Zip interface.

Instead you chose to make a useless snarky comment. Be better.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#207

Post by Akronymus »

cesarb wrote: post #37
dikei wrote: post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…

So thats why rarely the customer cant open one of the zip files we send over.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#208

Post by michaelcampbell »

ltbarcly3 wrote: post #204
starkrights wrote: post #152

Earlier quoted context omitted.

> a program that doesn’t matter anymore The rest of this comment has, though gratuitously snarky, a point, but I don’t think claiming that 7zip is irrelevant as an independent statement is even remotely coherent.

tar.zstd is superior in basically every way, open, and portable

It's fine that we run in different circles, but I have yet to see one of these in the wild.

Betamax was better, too.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#209

Post by sltkr »

ars wrote: post #191
sltkr wrote: post #180

Earlier quoted context omitted.

I think your mistake was to use --ultra without a compression level. I redid your experiments with rust-wasm-1.83.0-r0.apk: size perc c.time d.time uncompressed: 290072064 - - gzipped original: 105255109 36.29% - bzip2 -9: 107099379 36.92% 21.1s 11.0s bzip3 -b511: 73539847 25.35% 28.9s 32.0s xz --extreme -9: 71010672 24.48% 142.0s 3.1s lzip -9: 70964413 24.46% 173.5s 5.3s zstd --ultra -22: 48288499 16.64% 155.6s 0.4s…

That was an impressive result, so I tried it on a huge email inbox. uncompressed: 1512662084 xz --extreme -9: 508431572 12:47 zstd --ultra -21: 508432560 12:44 (-22 ran out of memory.) So at least by me zstd was identical to xz almost to the byte and the second.

It does really vary based on the data set.

If the email data is mostly text with markup (like HTML/XML), you might want to try bzip3 too.

It's also possible that a large part of your email is actually already-compressed binary data (like PDFs and images) possibly encoded in base-64. In that case it's likely that all tools are pretty good at compressing the text and headers, but can do little to compress the attachments, which would explain why the results you get are so close.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#210

Post by nick238 »

TkTech wrote: post #169
landl0rd wrote: post #160

Earlier quoted context omitted.

No, they're just not going to work with you.

Yep. Half the world's finances still spin on CSVs and FTP (no, not SFTP, FTP) If your customers request a format, that's the format you're using.

And if they don't request a format (or if you ask, and the response is "what's a format"), you need to use one that's 99.99% supported.
Post reply on HN