Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

7-Zip for Windows can now use more than 64 CPU threads for compression

7-zip.org

Post reply on HN
Print view View original

121–130 of 220 posts

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#121

Post by vpShane »

izzydata wrote: post #116

This may or may not be a relevant question, but does the terminology of "zip" have the same origin as the zip disk drive?

The ZIP file format was created in 1989 by Phil Katz for his PKZIP software.

The name 'ZIP' was chosen to convey the idea of compressing files into a smaller size, making them 'zip' together for easier storage and transfer. The term suggests speed and efficiency in compressing and uncompressing files.

Like zipping something up in a bag. You unzip the bag and have all of its contents, zip it up and all of its contents are in one bag (file).

7zip has always been the defacto-best, I'm sure there's other decent compression algorithms but 7z has always taken the cake for me.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#122

Post by soruly »

Szpadel wrote: post #106
sidewndr46 wrote: post #67

Earlier quoted context omitted.

What are you compressing with zstd? I had to do this recently and the "xz" utility still blows it away in terms of compression ratio. In terms of memory and CPU usage, zstd wins by a large margin. But in my case I only really cared about compression ratio

in my experience using zstd --long --ultra -22 gives marginally better compression ratio than xz -9 while being significantly faster

I think it depends on what you're compressing. I experimented with my data full of hex text xml files. xz -6 is both faster and smaller than zstd -19 by about 10%. For my data, xz -2 and zstd -17 achieve the same compressed size but xz -2 is 3 times faster than zstd -17. I still use xz for archive because I rarely needs to decompress them.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#123

Post by Night_Thastus »

izzydata wrote: post #115
Night_Thastus wrote: post #113

Earlier quoted context omitted.

7-zip is the de-facto tool on Windows and has been for a long time. It's more than fast and compressed enough for 99% of peoples use cases. It's not going anywhere anytime soon. The more likely thing to eat into its relevance is now that Windows has built-in basic support for zipping/unzipping EDIT: other formats*, which relegates 7-zip to more niche uses.

Is there something different about the built in zip context menu functionality now than before? I'm pretty sure you could convert something to a zip file since forever ago by right clicking any file.

It could support basic ZIP files, but only Windows 11 added support for 7-Zip (.7z), RAR (.rar), TAR, and TAR variants (like .tar.gz, .tar.bz2, etc).

That makes it 'good enough' for the vast majority of people, even if it's not as fast or fully-featured as 7-Zip.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#124

Post by Night_Thastus »

malfist wrote: post #119
Night_Thastus wrote: post #113

Earlier quoted context omitted.

7-zip is the de-facto tool on Windows and has been for a long time. It's more than fast and compressed enough for 99% of peoples use cases. It's not going anywhere anytime soon. The more likely thing to eat into its relevance is now that Windows has built-in basic support for zipping/unzipping EDIT: other formats*, which relegates 7-zip to more niche uses.

Windows has had built in zip/unzip since vista. 7zip is far superior (and the install base proves that)

As mentioned in another comment, zip support actually goes further back as far as '98, but only Windows 11 added support for handling other formats like RAR/7-Zip/.tar/.tar.gz/.tar.bz2/etc.

That allows it to be a default that 'just works' for most people without installing anything extra.

The vast majority of users don't care about the extra performance or functionality of a tool like 7-zip. They just need a way to open and send files and the Windows built-in tool is 'good enough' for them.

I agree that 7-zip is better, but most users simply do not care.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#125

Post by RealStickman_ »

krapht wrote: post #114
tiagod wrote: post #97

Earlier quoted context omitted.

What stops you from running updated zip/unzip on an old OS or on old hardware?

Nothing, but what stops you from using DEFLATE64? Installing new software has a real time and hassle cost, and how much time are you actually saving over the long run? It depends on your usage patterns.

Supporting old APIs and additional legacy ways of doing things has a real cost in maintenance.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#126

Post by zamadatix »

sidewndr46 wrote: post #73
zamadatix wrote: post #38

Earlier quoted context omitted.

> And x86 arguably didn't ship >64 hardware thread systems until then because NT didn't support it. If that were the case the above system wouldn't have needed 8 sockets. With NUMA systems the app needs to be scheduling group aware anyways. The difference here really appears when you have a single socket with more than 64 hardware threads, which took until ~2019 for x86.

Why would an application need to be NUMA aware on Linux? Most software I've ever written or looked at has no concept of NUMA. It works just fine.

The same reasons it would on macOS or Windows, most people just aren't writing software which needs to worry about having a single process running many hundreds of threads across 8 sockets efficiently so it's fine to not be NUMA aware. It's not that it won't run at all, a multi-socket system is still a superset of a single socket system, just it will run much more poorly than it could in such scenarios.

The only difference with Windows is a single processor group cannot contain more than 64 cores. This is why 7-Zip needed to add processor group support - even though a 96 core Threadripper represents as a single NUMA node the software has to request assignment to 2x48 processor groups, the same as if it were 2 NUMA nodes with 48 cores each, because of the KAFFINITY limitation.

Examples of common NUMA aware Linux applications are SAP Hana and Oracle RDBMS. On multi-socket systems it can often be helpful to run postgres and such via https://linux.die.net/man/8/numactl too, even if you're not quite the scale you need full NUMA awareness in the DB. You generally also want hypervisors to pass the correct NUMA topologies to guests as well. E.g. if you have a KVM guest with 80 cores assigned on a 2x64 Epyc host setup then you want to set the guest topology to something like 2x40 cores or it'll run like crap because the guest is sees it can schedule one way but reality is another.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#127

Post by Bender »

Night_Thastus wrote: post #113
d33 wrote: post #10

I worry that 7-Zip is going to lose relevance because lack of zstd support. zlib's performance is intolerable for large files and zlib-ng's SIMD implementation only helps here a bit. Which is a shame, because 7-Zip is a pretty amazing container format, especially with its encryption and file splitting capabilities.

7-zip is the de-facto tool on Windows and has been for a long time. It's more than fast and compressed enough for 99% of peoples use cases. It's not going anywhere anytime soon. The more likely thing to eat into its relevance is now that Windows has built-in basic support for zipping/unzipping EDIT: other formats*, which relegates 7-zip to more niche uses.

7-zip is the de-facto tool on Windows and has been for a long time.

Agreed. The only thing I think it has been missing is PAR support. I think they should consider incorporating one of the par2cmdline forks and porting that code to Windows as well so that it has recovery options similar to WinRAR. It's not used by everyone but that should deprecate any use cases for WinRAR in my opinion.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#128

Post by iamleppert »

malfist wrote: post #119
Night_Thastus wrote: post #113

Earlier quoted context omitted.

7-zip is the de-facto tool on Windows and has been for a long time. It's more than fast and compressed enough for 99% of peoples use cases. It's not going anywhere anytime soon. The more likely thing to eat into its relevance is now that Windows has built-in basic support for zipping/unzipping EDIT: other formats*, which relegates 7-zip to more niche uses.

Windows has had built in zip/unzip since vista. 7zip is far superior (and the install base proves that)

Windows unzip is so ungodly slow and terrible! Long live 7zip!

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#129

Post by mananaysiempre »

RealStickman_ wrote: post #125
krapht wrote: post #114

Earlier quoted context omitted.

Nothing, but what stops you from using DEFLATE64? Installing new software has a real time and hassle cost, and how much time are you actually saving over the long run? It depends on your usage patterns.

Supporting old APIs and additional legacy ways of doing things has a real cost in maintenance.

So does not supporting them, but the developer gets to externalize those.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#130

Post by wmil »

sidewndr46 wrote: post #70
jorvi wrote: post #25

Earlier quoted context omitted.

.. but 7-zip has a pretty terrible GUI? Hence why PeaZip is so popular, and J-Zip used to be before it was stuffed with adware.

If you're expecting a "mobile first" or similar GUI where most of the screen is dedicated to whitespace, basic features involves 7 or more mouse clicks and for some reason it all gets changed every ~6 months then yes the 7zip GUI is terrible. Desktop software usability peaked sometime in the late 90s, early 2000s. There's a reason why 7zip still looks like ~2004

When compared to it's contemporaries the 7-zip GUI is noticeably worse. Back in 2004 WinRar and WinZip were both clearly superior.
Post reply on HN