I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.
7-Zip for Windows can now use more than 64 CPU threads for compression
141–150 of 220 posts
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#142How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently. I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#143How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently. I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#144Earlier quoted context omitted.
What stops you from running updated zip/unzip on an old OS or on old hardware?
Nothing, but what stops you from using DEFLATE64? Installing new software has a real time and hassle cost, and how much time are you actually saving over the long run? It depends on your usage patterns.
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#145Earlier quoted context omitted.
Well, only a lunatic would use ZIP with anything but DEFLATE/DEFLATE64
there are A LOT of zip files using lzma in the wild. also, how about people learn to use updated software? should newer video compression technologies not be allowed in mkv/mp4. if you cant open it, well.. then stop using 90ies winzip
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#146Earlier quoted context omitted.
I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.
I don't know about, had a dicey situation recently where powershell's compress-archive couldn't handle archives >4GB and had to use 7zip. it is more reliable and you can ship 7za.exe or create self-extracting archives (wish those were more of a thing outside of the windows world).
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#147Earlier quoted context omitted.
I use ZSTD a ton in my programming work where efficiency matters. But for sharing files with other people, ZIP is still king. Even 7z or RAR is niche. Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger.
What are you compressing with zstd? I had to do this recently and the "xz" utility still blows it away in terms of compression ratio. In terms of memory and CPU usage, zstd wins by a large margin. But in my case I only really cared about compression ratio
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#148Earlier quoted context omitted.
Windows has had built in zip/unzip since vista. 7zip is far superior (and the install base proves that)
As mentioned in another comment, zip support actually goes further back as far as '98, but only Windows 11 added support for handling other formats like RAR/7-Zip/.tar/.tar.gz/.tar.bz2/etc. That allows it to be a default that 'just works' for most people without installing anything extra. The vast majority of users don't care about the extra performance or functionality of a tool like 7-zip. They just need a way to o…
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#149Earlier quoted context omitted.
> Everyone can open a ZIP file, and they don't really care if the file is a few MBs bigger. You can use ZSTD with ZIP files too! It's compression method 93 (see https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT which is the official ZIP file specification). Which reveals that "everyone can open a ZIP file" is a lie. Sure, everyone can open a ZIP file, as long as that file uses only a limited subset of the Z…
> Copyright (c) 1989 - 2014, 2018, 2019, 2020, 2022 Mostly it seems nutty that, after all these years, they’re still updating the zip spec instead of moving on to a newer format.
Re: 7-Zip for Windows can now use more than 64 CPU threads for compression
#150Earlier quoted context omitted.
Same thing with "WAV" files. There's at least 3 popular formats for the audio data out there.
More 'useful' one is webp. It has both a lossy and lossless compression algorithm, which have very different strengths and weaknesses. I think nearly every device supports reading both, but so many 'image optimization' libraries and packages don't - often just doing everything as lossy when it could be lossless (icons and what not).