Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

7-Zip for Windows can now use more than 64 CPU threads for compression

7-zip.org

Post reply on HN
Print view View original

161–170 of 220 posts

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#161

Post by homebrewer »

Am4TIfIsER0ppos wrote: post #91
redeeman wrote: post #81

Earlier quoted context omitted.

there are A LOT of zip files using lzma in the wild. also, how about people learn to use updated software? should newer video compression technologies not be allowed in mkv/mp4. if you cant open it, well.. then stop using 90ies winzip

mkv or mp4 with h264 and aac is good enough. mp3 is good enough. jpeg is good enough. zip with deflate is also good enough.

In the middle of San Francisco, with Silicon Valley level incomes, very possible. In the real world I still exchange files with users on rustic ADSL, where every megabyte counts. Many areas out there, in rural Mongolia or in the middle of Africa that's just got access to the internet, are even worse in that regard.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#162

Post by amelius »

fabiensanglard wrote: post #141

How does that work? You cannot write to disk before you know the compressed size. Or if you do you can use a data descriptor but you cannot write concurrently. I guess they buffer the compressed stream to RAM before writing to zip. If they want to keep their zip stable (always the same output given the same input), they also need to keep it a bit longer than necessary in RAM.

Maybe Windows allows a file to be constructed quickly from parts that are not necessarily multiples of the blocksize. Maybe they have a fast api for writing multiple files and then turning it into a single file. POSIX doesn't allow that, but it is quite old.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#163

Post by danudey »

xxs wrote: post #107
throw0101d wrote: post #79

Earlier quoted context omitted.

> You can use ZSTD with ZIP files too! Support for which was added in 2020: > On 15 June 2020, Zstandard was implemented in version 6.3.8 of the zip file format with codec number 93, deprecating the previous codec number of 20 as it was implemented in version 6.3.7, released on 1 June.[36][37] * https://en.wikipedia.org/wiki/Zstd#Usage So I'm not sure how widely deployed it would be.

Most linux distributions have zip support with zstd.

The `zip` command on Ubuntu is 6.0, which was released in 2009 and does not support zstd. It does support bzip2 though!

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#164

Post by monocasa »

p_ing wrote: post #138
arp242 wrote: post #65

Earlier quoted context omitted.

> Computers didn’t exceed 64 logical processors per system until around 2014. Server systems were available with that since at least the late 90s. Server systems with >10 CPUs were already available in the mid-90s. By the early-to-mid 90s it was pretty obvious that was only going to increase and that the 64-CPU limit was going to be a problem down the line. That said, development of NT started in 1988, and it may hav…

"Server systems" but not server systems that Microsoft targeted. NT4 Enterprise Server (1996) only supported up to 8 sockets (some companies wrote their own HAL to exceed that limit). And 8 sockets was 8 threads with no NUMA back then, not something that would have been an issue for the purposes of this discussion.

Microsoft was absolutely wanting to target large servers at the time. They were actively trying to kill off the vendor unices in the 90s.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#166

Post by danudey »

LegionMammal978 wrote: post #150
martinald wrote: post #111

Earlier quoted context omitted.

More 'useful' one is webp. It has both a lossy and lossless compression algorithm, which have very different strengths and weaknesses. I think nearly every device supports reading both, but so many 'image optimization' libraries and packages don't - often just doing everything as lossy when it could be lossless (icons and what not).

It's similarly annoying how many websites take the existence of the lossy format as a license to recompress all WebP uploads, or sometimes other filetypes converted to WebP, even when it causes the filesize to increase. It's like we're returning to ye olden days of JPEG artifacts on every screenshot.

I was thinking about this with YouTube as an example. A lot of people complain about the compression on YouTube videos making things look awful, but I bet there's a reasonable number of high-end content creators out there who would run a native(-ish, probably Electron) app on their local system to do a higher-quality encoding to YouTube's specifications before uploading.

In many (most?) cases, it's possible to get better compression and higher quality if you're willing to spend the CPU cycles on it, meaning that YouTube could both reduce their encoding load and increase quality at the same time, and content creators could put out better quality videos that maintain better detail.

It would certainly take longer to upload the multiple multiple versions of everything, and definitely it would take longer to encode, but it would also ease YouTube's burden and produce a better result.

Ah well, a guy can dream.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#167

Post by ale42 »

kevinventullo wrote: post #94
mauvehaus wrote: post #63

Earlier quoted context omitted.

> I wonder when we’ll start calling them “Office files”. Probably around the same time the save icon becomes something other than a 3 1/2" floppy disk.

Nowadays I’ve noticed fewer applications have a save icon at all, relying instead on auto-save.

And some only save to the cloud, whence a cloud icon with an arrow. (Not that I like that, but... that's what we get)

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#168

Post by danudey »

6SixTy wrote: post #149
justin66 wrote: post #51

Earlier quoted context omitted.

> Copyright (c) 1989 - 2014, 2018, 2019, 2020, 2022 Mostly it seems nutty that, after all these years, they’re still updating the zip spec instead of moving on to a newer format.

.7z and .tar.* have existed for at least 20 years now, but you are unlikely to see a wild 7z file and .tar.* is isolated to the UNIX space

Tar files also have the miserable limitation of having no index; this means that to extract an individual file requires scanning through the entire archive until you find it, and then continuing to scan through the rest of the archive because a tar file can have the same file path added multiple times.

That makes them useful for transferring an entire set of files that someone will want all or none of, e.g. source code, but terrible for a set of files that someone might want to access arbitrary files from.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#169

Post by TkTech »

landl0rd wrote: post #160
redeeman wrote: post #154

Earlier quoted context omitted.

if they want to open certain files, they will update

No, they're just not going to work with you.

Yep. Half the world's finances still spin on CSVs and FTP (no, not SFTP, FTP) If your customers request a format, that's the format you're using.

Re: 7-Zip for Windows can now use more than 64 CPU threads for compression

#170

Post by philipwhiuk »

redeeman wrote: post #155
1over137 wrote: post #87

Earlier quoted context omitted.

>how about people learn to use updated software? How about software developers learn to keep software working on old OSes and old hardware?

what software doesnt support OSs that are in active SECURITY support?

https://arstechnica.com/gadgets/2023/07/windows-95-98-and-ot...
Post reply on HN