Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UPX – Ultimate Packer for Executables

upx.github.io

Post reply on HN
Print view View original

51–60 of 82 posts

Re: UPX – Ultimate Packer for Executables

#51

Post by KGIII »

userbinator wrote: post #47
hannob wrote: post #2

Surprised this still exists. A little walk down memory lane: I once ran the exe mailing list for exe packers and protection tools. There was a whole scene of people in the 90s writing such tools and writing unpackers and removal tools for such things. UPX was one of the later ones that still existed when most of this scene vanished.

The demoscene continues to make extensive use of packers, more for achieving even better compression than for protection against RE: https://in4k.github.io/wiki/exe-packers-tweakers-and-linkers (Incidentally, these advanced packers also tend to frustrate RE to some extent, since the same tricks they use to increase compression ratios can often greatly confuse RE tools.)

"RE" is reverse engineering, maybe?

Re: UPX – Ultimate Packer for Executables

#52

Post by KGIII »

pilif wrote: post #49
userbinator wrote: post #46

Earlier quoted context omitted.

I'm surprised to hear that --- I can and have seen it happening with more advanced/obscure/protective packers, but UPX is so common and very easily unpacked (and thus scanned by AVs) that I'd say any AV which gets confused by UPX is not worth using at all.

And yet... in our case, the false-positive rate went from about one a month to one a year when we stopped using UPX. For a binary that didn’t change, mind you. You'd think that after reporting a false positive once, an AV vendor would whitelist the hash of the binary, but no. Some of them were re-detecting malware time and time again. Until we stopped using UPX.

Could UPX put something in the header that said something akin to, 'I am not a signifier of malware, perform your check on the internal contents instead.'

Then AV companies could see that and not flag it as malware unless they had additional reason to think it was.

That doesn't seem like it'd be terribly difficult but there's a good chance I'm missing something.

Re: UPX – Ultimate Packer for Executables

#53

Post by jlebar »

smegel wrote: post #39
beagle3 wrote: post #31

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (ev…

> you stop the OS from being able to page out your executable (rarely a big loss) Why is this exactly?

I think what OP meant is, if you have a page that's backed by disk, then when you're low on RAM, the OS can simply drop that page -- it's sitting there on disk if you ever need it back. But if you have a page that's not backed by disk, then you have to write it out to disk before you can drop it.

Re: UPX – Ultimate Packer for Executables

#54

Post by opk »

The one time I came across upx, it was used on some malware. It was on a programme named gnome-pty-helper in a user's .config directory that was installed in cron and set to phone home to some locations that were stored in clear once upx had been used to unpack it.

Years back I used gzexe and also some pkzip based thing on DOS. On a modern system, you're better of enabling filesystem level compression which also won't break OS paging if the executable is run more than once.

Re: UPX – Ultimate Packer for Executables

#55

Post by takeda »

KGIII wrote: post #52
pilif wrote: post #49

Earlier quoted context omitted.

And yet... in our case, the false-positive rate went from about one a month to one a year when we stopped using UPX. For a binary that didn’t change, mind you. You'd think that after reporting a false positive once, an AV vendor would whitelist the hash of the binary, but no. Some of them were re-detecting malware time and time again. Until we stopped using UPX.

Could UPX put something in the header that said something akin to, 'I am not a signifier of malware, perform your check on the internal contents instead.' Then AV companies could see that and not flag it as malware unless they had additional reason to think it was. That doesn't seem like it'd be terribly difficult but there's a good chance I'm missing something.

You're talking as if the AV companies don't know hat UPX is.

They know it very well, but adding code to do decompression while performing scan is more complex and will surely reduce performance.

If the AV is already slow, they might decide to just label any UPX binary, since (let's not lie) most malware will be compressed with UPX or other tools.

Re: UPX – Ultimate Packer for Executables

#56

Post by takeda »

foobarchu wrote: post #38
beagle3 wrote: post #31

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (ev…

I'm curious, why did you feel the need to call out go specifically? Does this apply more to go than other source languages?

If you look at other responses large number of them actually mentions that UPX works really with go and docker, I believe it was response to that.

Re: UPX – Ultimate Packer for Executables

#57

Post by beagle3 »

foobarchu wrote: post #38
beagle3 wrote: post #31

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (ev…

I'm curious, why did you feel the need to call out go specifically? Does this apply more to go than other source languages?

I wrote "or any other binaries, really". It's just that

(a) go compiles static binaries which makes UPX especially effective, unlike e.g. common C and C++ projects with their hundreds of .so/.dlls ; Delphi/FPC and Nim or the other two ecosystems that share this trait, but neither is as common as go.

(b) it's not good for non-static binaries, that is C#, Java, Python have no benefit from this.

(c) At the time I posted it, there were already 3 or 4 posts extolling the virtues of compressing Go executables.

Re: UPX – Ultimate Packer for Executables

#58

Post by beagle3 »

smegel wrote: post #39
beagle3 wrote: post #31

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (ev…

> you stop the OS from being able to page out your executable (rarely a big loss) Why is this exactly?

Indeed, as jlebar notes - since the actual executable code (the thing that the CPU executes) does not exist directly in the executable file, the OS will have to write it out to the swap if it needs the memory (unlike uncompressed files, where it just reuses the memory and later reloads from the executable file).

It is rarely a big loss, because executables that are in use tend to remain in memory if the program is actually active. If you have a 300MB demon that sleeps, though, you will likely notice a swap out to magnetic disk.

Re: UPX – Ultimate Packer for Executables

#59

Post by FooBarWidget »

beagle3 wrote: post #31

People who compress their go binaries (or any other binaries, really) - please be aware that thus doing, you stop the OS from being able to page out your executable (rarely a big loss), and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). If there's only one copy of a program running, it won't matter - but if you are running hundreds of copies (ev…

> and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable).

I don't think people care about that nowadays, seeing how popular Docker containers are. I think Docker containers already make it so that you cannot share executable memory between different containers because each one runs in its private namespace.

Re: UPX – Ultimate Packer for Executables

#60

Post by gordaco »

lwerdna wrote: post #4
hannob wrote: post #2

Surprised this still exists. A little walk down memory lane: I once ran the exe mailing list for exe packers and protection tools. There was a whole scene of people in the 90s writing such tools and writing unpackers and removal tools for such things. UPX was one of the later ones that still existed when most of this scene vanished.

I think some of that still exists, but the goal is to to evade anti-virus instead of compressing and deterring RE: https://hackforums.net/forumdisplay.php?fid=299

I remember cases where the AV successfully detected the upxed executable, but not the original, because upx was so widespread that the most common version of the infected file was upxed.
Post reply on HN