Live data from Hacker News

7-Zip for Windows can now use more than 64 CPU threads for compression

7-zip.org

141–150 of 220 posts

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#141
How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently.

I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#142

How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently. I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.

I think you get different compressed files depending on how many threads you use to compress

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#143

How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently. I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.

[deleted]

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#144
post #114
post #97

Earlier quoted context omitted.

What stops you from running updated zip/unzip on an old OS or on old hardware?

Nothing, but what stops you from using DEFLATE64? Installing new software has a real time and hassle cost, and how much time are you actually saving over the long run? It depends on your usage patterns.

what stops you from installing win95 and winzip?

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#145
post #41

Earlier quoted context omitted.

Well, only a lunatic would use ZIP with anything but DEFLATE/DEFLATE64

there are A LOT of zip files using lzma in the wild. also, how about people learn to use updated software? should newer video compression technologies not be allowed in mkv/mp4. if you cant open it, well.. then stop using 90ies winzip

No. You can't get people to use updated software. You can't get a number of people to update past windows 7. This has been and will likely remain a persistent issue, and it's sure not one you're going to fix. All it will do is limit your ability to work with people. This isn't a hill on which you should die.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#146
post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

I don't know about, had a dicey situation recently where powershell's compress-archive couldn't handle archives >4GB and had to use 7zip. it is more reliable and you can ship 7za.exe or create self-extracting archives (wish those were more of a thing outside of the windows world).

I understand that security has to compromise for the real world, but a self-extracting archive is possibly one of the worst things one could use in terms of security.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#147
post #29

Earlier quoted context omitted.

I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.

What are you compressing with zstd? I had to do this recently and the "xz" utility still blows it away in terms of compression ratio. In terms of memory and CPU usage, zstd wins by a large margin. But in my case I only really cared about compression ratio

I usually see zstd on max settings outperform xz on speed and very slightly on compression (though that's a tiny difference).

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#148

Earlier quoted context omitted.

Windows has had built in zip/unzip since vista. 7zip is far superior (and the install base proves that)

As mentioned in another comment, zip support actually goes further back as far as '98, but only Windows 11 added support for handling other formats like RAR/7-Zip/.tar/.tar.gz/.tar.bz2/etc. That allows it to be a default that 'just works' for most people without installing anything extra. The vast majority of users don't care about the extra performance or functionality of a tool like 7-zip. They just need a way to o…

Windows zip is not in fact good enough. I've run into weird, buggy behavior, hanging on extract, all sorts of nonsense. I can see the argument that a universally-adopted solution is better, but that's different from windows just not working.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#149
post #37

Earlier quoted context omitted.

> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…

> Copyright (c) 1989 - 2014, 2018, 2019, 2020, 2022 Mostly it seems nutty that, after all these years, they’re still updating the zip spec instead of moving on to a newer format.

.7z and .tar.* have existed for at least 20 years now, but you are unlikely to see a wild 7z file and .tar.* is isolated to the UNIX space

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#150

Earlier quoted context omitted.

Same thing with "WAV" files. There's at least 3 popular formats for the audio data out there.

More 'useful' one is webp. It has both a lossy and lossless compression algorithm, which have very different strengths and weaknesses. I think nearly every device supports reading both, but so many 'image optimization' libraries and packages don't - often just doing everything as lossy when it could be lossless (icons and what not).

It's similarly annoying how many websites take the existence of the lossy format as a license to recompress all WebP uploads, or sometimes other filetypes converted to WebP, even when it causes the filesize to increase. It's like we're returning to ye olden days of JPEG artifacts on every screenshot.
Post reply on HN