Live data from Hacker News

7-Zip for Windows can now use more than 64 CPU threads for compression

7-zip.org

41–50 of 220 posts

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#41
post #37
post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…

Well, only a lunatic would use ZIP with anything but DEFLATE/DEFLATE64

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#42
post #25
post #13

Earlier quoted context omitted.

Not that many people care about zstd; I would assume most 7-zip users care about the convenience of the gui.

.. but 7-zip has a pretty terrible GUI? Hence why PeaZip is so popular, and J-Zip used to be before it was stuffed with adware.

> .. but 7-zip has a pretty terrible GUI?

Since you're asking, the answer is no. 7-Zip has an efficient and elegant UI.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#43

Earlier quoted context omitted.

> And x86 arguably didn't ship >64 hardware thread systems until then because NT didn't support it. If that were the case the above system wouldn't have needed 8 sockets. With NUMA systems the app needs to be scheduling group aware anyways. The difference here really appears when you have a single socket with more than 64 hardware threads, which took until ~2019 for x86.

There were single image systems with hundreds of cores in the late 90s and thousands of cores in the early 2000s. I absolutely stand by the fact that Intel and AMD didn't pursue high core count systems until that point because they were so focused on single core perf, in part because Windows didn't support high core counts. The end of Denmark scing forced their hand and Microsoft's processor group hack.

[deleted]

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#44
post #37
post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…

> new Office files

I know what you mean, I’m not being pedantic, but I just realized it’s been 19 years. I wonder when we’ll start calling them “Office files”.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#45

Earlier quoted context omitted.

> And x86 arguably didn't ship >64 hardware thread systems until then because NT didn't support it. If that were the case the above system wouldn't have needed 8 sockets. With NUMA systems the app needs to be scheduling group aware anyways. The difference here really appears when you have a single socket with more than 64 hardware threads, which took until ~2019 for x86.

There were single image systems with hundreds of cores in the late 90s and thousands of cores in the early 2000s. I absolutely stand by the fact that Intel and AMD didn't pursue high core count systems until that point because they were so focused on single core perf, in part because Windows didn't support high core counts. The end of Denmark scing forced their hand and Microsoft's processor group hack.

Do you have anything to say regarding NUMA for the 90s core counts though? As I said, it's not enough that there were a lot of cores - they have to be monolithically scheduled to matter. The largest UMA design I can recall was the CS6400 in 1993, to go past that they started to introduce NUMA designs.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#46

Earlier quoted context omitted.

> And x86 arguably didn't ship >64 hardware thread systems until then because NT didn't support it. If that were the case the above system wouldn't have needed 8 sockets. With NUMA systems the app needs to be scheduling group aware anyways. The difference here really appears when you have a single socket with more than 64 hardware threads, which took until ~2019 for x86.

There were single image systems with hundreds of cores in the late 90s and thousands of cores in the early 2000s. I absolutely stand by the fact that Intel and AMD didn't pursue high core count systems until that point because they were so focused on single core perf, in part because Windows didn't support high core counts. The end of Denmark scing forced their hand and Microsoft's processor group hack.

AMD and Intel were focused on single core performance, because personal desktop computing was the bigger business until around mid to late 2000s.

Single core performance is really important for client computing.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#47
post #37
post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…

You can and I've done it… but you can't expect anything to be able to decompress it unless you wrote it yourself.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#48
post #10

I worry that 7-Zip is going to lose relevance because lack of zstd support. zlib's performance is intolerable for large files and zlib-ng's SIMD implementation only helps here a bit. Which is a shame, because 7-Zip is a pretty amazing container format, especially with its encryption and file splitting capabilities.

You are looking for 7-Zip Zstd: https://github.com/mcmilk/7-Zip-Zstd

I don't know what your use case is, but it seems to be quite a niche.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#49

Earlier quoted context omitted.

There were single image systems with hundreds of cores in the late 90s and thousands of cores in the early 2000s. I absolutely stand by the fact that Intel and AMD didn't pursue high core count systems until that point because they were so focused on single core perf, in part because Windows didn't support high core counts. The end of Denmark scing forced their hand and Microsoft's processor group hack.

AMD and Intel were focused on single core performance, because personal desktop computing was the bigger business until around mid to late 2000s. Single core performance is really important for client computing.

They were absolutely interested in the server market as well.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#50
post #13
post #10

I worry that 7-Zip is going to lose relevance because lack of zstd support. zlib's performance is intolerable for large files and zlib-ng's SIMD implementation only helps here a bit. Which is a shame, because 7-Zip is a pretty amazing container format, especially with its encryption and file splitting capabilities.

Not that many people care about zstd; I would assume most 7-zip users care about the convenience of the gui.

I just hope that the recipient will be able to open the file without too much difficulty. I am willing to sacrifice a few megabytes if necessary.
Post reply on HN