Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UPX – Ultimate Packer for Executables

upx.github.io

Post reply on HN
Print view View original

71–80 of 82 posts

Re: UPX – Ultimate Packer for Executables

#71

Post by badsectoracula »

squarefoot wrote: post #67
analognoise wrote: post #50

Earlier quoted context omitted.

I've done the same thing! Shout out to Lazarus/FPC!!!

I've seen some posts about it over here recently and believe the critical users/developers mass to make it a successful dev system is achieved, however it still has to struggle to be taken into consideration in some contexts for not using a major language. If it allowed to build GUIs for C++ (QT/*TK and other builders are not even close to its usability level) it would probably become mainstream in a week.

Sometimes i wonder how hard it would be to make some C++ Builder-like modifications to Clang (like Embarcadero did for newer C++ Builder versions) to allow it use Free Pascal objects directly for people who really want to avoid Pascal.

Of course to do it properly it'd need:

* A modified Clang (or other C++ compiler) that can use .ppu files with the necessary C++ language extensions for properties, callbacks, sets, enhanced RTTI, etc

* A C/C++ library that uses the Free Pascal RTL for all memory operations

* Lazarus' CodeTools to add C++ support for automatically creating missing event handler code (and removing unnecessary code), handling syntax completion, code completion for missing identifiers, property getters/setters and private fields, inherited fields, etc to the same standard as the Free Pascal code

* All the involved teams to agree to play nice with each other :-P

Also if such a thing would be done, judging from what most Lazarus and FPC devs do so far, it'd probably be done in a way that is as compatible with C++ Builder as possible.

TBH i don't really hold my breath, but who knows, weird stuff has happened before in both FPC and Lazarus :-P

Re: UPX – Ultimate Packer for Executables

#72

Post by icebraining »

pgeorgi wrote: post #69
beagle3 wrote: post #58

Earlier quoted context omitted.

Indeed, as jlebar notes - since the actual executable code (the thing that the CPU executes) does not exist directly in the executable file, the OS will have to write it out to the swap if it needs the memory (unlike uncompressed files, where it just reuses the memory and later reloads from the executable file). It is rarely a big loss, because executables that are in use tend to remain in memory if the program is ac…

Also if you have the same executable running several times on a machine, they share the RAM for the code (read-only pages, which should be most of them). That doesn't work for UPX because each execution decompresses anew, which makes it a "new executable" from the OS' point of view. The only thing that would help is kernel space merging, but that's really only activated for some virtual machines.

that's really only activated for some virtual machines.

And even there, it's a security threat, since it enables cross-VM timing and other attacks.

Re: UPX – Ultimate Packer for Executables

#73

Post by gundul »

majewsky wrote: post #65
gundul wrote: post #63

Just to chip in on my experience on this matter. I use UPX as one of the RE defense method on a couple of Delphi based software we build which our customer runs regularly on their servers. One of the challenge is some A/V throws a false positive upon checking the result files. Somehow this became no longer an issue after applying code-signing to the UPX output executables.

> RE defense method How does UPX defend against reverse engineering? The binary literally contains the code to reverse the UPX compression (otherwise it couldn't run), and I'd expect all antiviruses to be able to unpack UPX executables.

It's just for adding obscurity, hence only one of the methods being in use.

Re: UPX – Ultimate Packer for Executables

#74

Post by [deleted] »

bruno2223 wrote: post #19
slezyr wrote: post #16

Earlier quoted context omitted.

That's interesting. The UPX string is most likely a name of section in PE file. It's first UPX string you will find in the file. How did UPX loader managed to find the section in which packed content is stored? UPD. It's REALLY easy to "hack" this protection. You simply need to attach a debugger and you will see unprotected exe file in the memory. There are tools to convert loaded unprotected exe file into regular ex…

I dunno know. But this method worked for years!

[deleted]

Re: UPX – Ultimate Packer for Executables

#75

Post by userbinator »

takeda wrote: post #55
KGIII wrote: post #52

Earlier quoted context omitted.

Could UPX put something in the header that said something akin to, 'I am not a signifier of malware, perform your check on the internal contents instead.' Then AV companies could see that and not flag it as malware unless they had additional reason to think it was. That doesn't seem like it'd be terribly difficult but there's a good chance I'm missing something.

You're talking as if the AV companies don't know hat UPX is. They know it very well, but adding code to do decompression while performing scan is more complex and will surely reduce performance. If the AV is already slow, they might decide to just label any UPX binary, since (let's not lie) most malware will be compressed with UPX or other tools.

If the AV is already slow, they might decide to just label any UPX binary, since (let's not lie) most malware will be compressed with UPX or other tools.

IMHO an AV that doesn't know how to unpack UPX is almost like an AV that doesn't know how to unpack ZIP or RAR... and yet they universally do the latter.

You'd think that after reporting a false positive once, an AV vendor would whitelist the hash of the binary, but no. Some of them were re-detecting malware time and time again. Until we stopped using UPX.

I have a feeling that your false positives are caused by the fact that UPX (and other compressors) naturally create very high-entropy files, and AVs which do signature-type comparisons would like to reduce signature length as much as possible, so they also choose very high-entropy portions of malware to be as distinctive as possible while remaining short; but that also increases the chances of such sequences being found in other benign high-entropy files.

I'm almost willing to bet that your re-detections are not detecting the same malware, but new ones' signatures as the AV vendor adds them --- which coincidentally happens to match some other high-entropy portion of your binary.

Then again, the quest for speed and high detection rates (while false positive rates seem to be less of a concern) among AV vendors has lead to some massively embarrassing mistakes, like considering the mere existence of a path as detection of malware:

https://www.f-secure.com/weblog/archives/00002133.html

(The original article with the ridiculous claims has sadly vanished, but the Internet Archive remembers...)

Re: UPX – Ultimate Packer for Executables

#76

Post by ptman »

beagle3 wrote: post #61
FooBarWidget wrote: post #59

Earlier quoted context omitted.

> and also to be unable to share executable pages (not a huge loss for a 2MB executable, a huge loss for a 100MB executable). I don't think people care about that nowadays, seeing how popular Docker containers are. I think Docker containers already make it so that you cannot share executable memory between different containers because each one runs in its private namespace.

Do namespaces separate file system caches? That would surprise me.

IIRC it depends on which storage backend for docker you use: https://web.archive.org/web/20170405122924/https://developer...

Re: UPX – Ultimate Packer for Executables

#77

Post by davexunit »

jamesrom wrote: post #41
davexunit wrote: post #40

Earlier quoted context omitted.

What a malicious thing to do. Computer users deserve to be able to inspect the code running on their machines.

Of course you can inspect it. Understanding it, however...

The user already doesn't have the source. Obfuscating the binary is just hostile.

Re: UPX – Ultimate Packer for Executables

#78

Post by JohnTHaller »

PortableApps.com used UPX for most open source releases up until a couple years ago. We stopped due to antivirus false positives combined with the fact that most folks have more space for their apps. We still make available the tool we use called PortableApps.com AppCompactor. It provides a simple GUI to use UPX on a whole directory and sub-directories. Plus it can optionally recompress JAR and ZIP files using 7-Zip. If it's useful to you, you can grab it here: https://portableapps.com/apps/utilities/portableapps.com_app...

Re: UPX – Ultimate Packer for Executables

#79

Post by mitchty »

mayank wrote: post #36
mitchty wrote: post #8

Earlier quoted context omitted.

50MiB to 6MiB The binaries are mostly stuff like pandoc and compiled statically so that I can run them anywhere. Nothing too special. Its not technically needed, but it makes network transfer faster and in general thats good enough. Its not really intended to reduce disk space really, just more a way to make things more manageable.

Curious — which network transfer protocol are you using that doesn’t support compression on the fly? And if its just for transfer, why not gzip instead?

2GiB quotas on home directories is another reason, but explaining it all on hn isn't really a goal of mine. Suffice it to say there are a confluence of issues. As to network compression via ssh say versus compression of the binary, time it you might be surprised at the difference.

Re: UPX – Ultimate Packer for Executables

#80

Post by mitchty »

vetinari wrote: post #13
mitchty wrote: post #6

Earlier quoted context omitted.

I use it for static binaries I use where I don't care about startup times being slower. You'd be suprised at how much of an elf binary is all 0's.

Compressing them this way however makes situation worse for memory manager. If you use uncompressed (or transparently compressed by the filesystem) binary, your process has mmaped the memory pages, which can be discarded and then reloaded, as needed. If you use self-extractor, your process has dirty pages that it itself wrote, that cannot be discarded, but must be moved to swap if needed. The more you use the same ex…

For the binaries used this isn't really a concern. Most are cli binaries run every so often.
Post reply on HN